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State of Florida 
Department of Education 
Bureau of Contracts, Grants and Procurement Management Services 

325 West Gaines Street 
332 Turlington Building 
Tallahassee, Florida 32399-0400 



INVITATION TO NEGOTIATE REGISTRATION 
COMPLETE AND RETURN THIS FORM 

Reply Number: ITN 2014-47 

Title: Florida Standards Assessments 

Date & Time Reply Due: December 12, 2013 @ 2:00 PM, EASTERN TIME (ET) 



Potential Respondents should notify the Florida Department of Education, Bureau of Contracts, Grants and 
Procurement ManagementServices by returning this Intentto Submit Reply Form as soon as possible after 
downloading. Complete the information below and send this sheet only to below e-mail address 
(preferred). Form can also be faxed to fax number (850) 245-0719, or mailed to 325 West Gaines Street, 
332 Turlington Building, Tallahassee, Florida 32399-0400. 



Company Name: 
Contact Person: 



Address: 

City, State, Zip: 

Telephone: ( ) Fax Number: J \ 

Internet E-Mail Address: 



Signed: Date: 

For further information on this process, you may contact Fran Shewan - fran.shewan@fldoe.org at (850) 
245-9884. 

(Revised 10/06/2012) 
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STATE OF FLORIDA 
DEPARTMENT OF EDUCATION 
INVITATION TO NEGOTIATE 

FLORIDA STANDARDS ASSESSMENTS 

BID NUMBER: ITN 2014-47 

SECTION 1- OVERVIEW 
SECTION 1 -INSTRUCTIONS 

1.0 GENERAL INSTRUCTIONS TO RESPONDENT 

This section contains instructions explaining the solicitation process and the actions necessary to respond. 
General Instructions to Respondent (Form PUR 1001 - incorporated herein by reference) is a downloadable 
document which must be downloaded for review. This document need not be returned with the Respondent's 
Reply. Form PUR 1001 may be accessed at http://dms.myflorida.com/business operations/state purchasing 
under "Documents, Forms, References and Resources". 

In the event of any conflict between Form PUR 1001 and other instructions provided in this document, the 
additional instructions in this document shall take precedence over the Form PUR 1001 unless the conflicting 
term is required by any section of the Florida Statutes (F.S.), in which case the statutory requirements shall 
take precedence. 

It is entirely the respondent's responsibility to examine this ITN, to confirm that the Department's requirements 
are clearly stated, and to submit its reply in a timely, complete, and procedurally correct manner. The 
products and services described in this ITN will be procured in accordance with Chapter 287, Florida Statutes 
(F.S.), and Chapter 60A-1, Florida Administrative Code (FAC). 

THE DEPARTMENT HAS CHOSEN TO USE THE ITN FORMAT FOR THIS PROCUREMENT BECAUSE IT 
WANTS VENDORS TO PROPOSE THE BEST METHOD FOR ACHIEVING THE GOAL OF THIS ITN AND 
SOLVING THE PROBLEM STATED HEREIN. THEREFORE, ALTHOUGH THE ITN MAY USE 
MANDATORY WORDS LIKE "SHALL," "WILL," OR "MUST," AND MAY DEFINE CERTAIN ITEMS AS 
REQUIREMENTS, THE DEPARTMENT RESERVES THE RIGHT, IN ITS DISCRETION, TO WAIVE ANY 
DEVIATIONS FROM THESE PROVISIONS AND RESOLVE ANY ISSUES IN THE NEGOTIATION PHASE. 
THEREFORE, THE DEPARTMENT RESERVES THE RIGHT TO REVIEW THE ENTIRE REPLY TO 
DETERMINE IF IT ACHIEVES A LEVEL OF COMPETENCY WORTHY OF FURTHER NEGOTIATIONS; 
REGARDLESS OF WHETHER INDIVIDUAL REQUIREMENTS HAVE BEEN ADDRESSED OR NOT. 
HOWEVER, VENDORS THAT FAIL TO PROVIDE SIGNIFICANT PORTIONS OF THE SOLUTION OR 
ADDRESS SIGNIFICANT PORTIONS OF THE PROCUREMENT MAY STILL BE DEEMED 
NONRESPONSIVE. IN ADDITION, THERE IS NO GUARANTEE THAT SUCH DEVIATIONS WILL BE 
DEEMED IN THE STATE'S BEST INTEREST OR ANY REPLY CONTAINING THOSE DEVIATIONS 
PARTICIPATE IN THE NEGOTIATIONS. USE OF THE TERMS "SHALL," "WILL," AND "MUST" INDICATE 
THE DEPARTMENT'S INITIAL VIEW OF THE VALUE OF SUCH ITEMS. VENDORS HAVE THE 
OPPORTUNITY TO SUGGEST ALTERNATIVES IN THE ITN PROCESS, BUT THERE IS NO GUARANTEE 
THAT THE DEPARTMENT WILL AGREE THE DEVIATIONS ARE IN ITS BEST INTEREST OR CREATE 
THE BESTVALUE FOR THE STATE. 
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SECTION 2 - CONTRACT CONDITIONS 



2.0 GENERAL CONTRACT CONDITIONS 

Standard terms and conditions that will apply to the contract which results from the solicitation event are 
provided in this section. General Contract Conditions (Form PUR 1000 - incorporated herein by reference) is 
a downloadable document which must be downloaded for review. This document need not be returned with 
the Respondent's Reply. Form PUR 1000 may be accessed at 
http://dms.myflorida.com/business operations/state purchasing under "Documents, Forms, References and 
Resources". 

In the event of any conflict between the PUR 1000 form and any other Special Conditions, the Special 
Conditions shall take precedence over the PUR 1000 form unless the conflicting term in the PUR form is 
required by any section of the F.S., in which case the statutory requirements shall take precedence. 



SECTION 3 -INTRODUCTION 

3.0 INTENT 

The State of Florida Department of Education (hereinafter referred to as the "Department") is soliciting written 
replies from qualified vendors to establish a term contract of which the term is anticipated to begin upon 
execution of the contract and be effective for 36) months thereafter. Award(s) will be made to the responsible 
and responsive vendor that the Department determines will provide the best value to the state. 

The resulting contract(s) may be renewed for up to three additional 12 month terms, see Form PUR 1000 for 
renewal requirements. 

Although this ITN may result in the award of multiple contracts, the term "Contractor" is used throughout this 
ITN and refers to each contractor providing the required products and services for each grade and subject for 
which the respective contract is awarded. 

3.1 PURPOSE 

The purpose of this Invitation to Negotiate (ITN) is to solicit replies for activities related to the implementation 
of computer-based and paper-based statewide summative assessments aligned to the Florida Standards for 
administrations beginning in spring 2015 and continuing through 2016-17 for the base contract with three one- 
year optional renewals for the administrations in 2017-18, 2018-19, and 2019-20. Henceforth, in this 
document, this test shall be called the FSA (Florida Standards Assessment) 

In accordance with Governor Rick Scott's Executive Order, dated September 23, 2013, this ITN is seeking 
Replies that: 

A. P rovides timely and informative reports of results; 

B. Includes costs that are in line with current assessment costs; 

C. Ensures that testing time for students is not significantly different from current assessments; 

D. P rovides for summative assessments to occur as close as possible to the end of the school year; 

E. Measures student mastery of the standards taught, including comparability to other states; 

F. Includes test quality metrics that are as rigorous as current assessments; 

G. Provides results that can be used in conjunction with Florida's school accountability system; 

H. Requires technology parameters that are defined and can be supported, including appropriate 
accommodations for exceptional students. 
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This ITN addresses four broad assessment components as described below. Respondents may submit bids 
on any individual assessment component (A, B, C or D), any group of assessment components, or to provide 
all assessment components (A, B, C and D): 

A. Computer-based and paper-based assessments in English Language Arts/ Literacy in Grades 3-8 

B. Computer-based and paper-based assessments in Mathematics in Grades 3-8 

C. Computer-based and paper-based assessments in English Language Arts/ Literacy in Grades 9-11 

D. Computer-based and paper-based End-of-Course (EOC) assessments in Algebra 1, Geometry, and 
Algebra 2 

A preferred schedule and plan for transition to computer-based testing is included in Appendix A. However, 
depending on the final award for the individual assessments, the Department may vary when tests will be 
administered as computer-based or paper-based. The final determination of which tests will be computer- 
based or paper-based is at the Department's discretion. 

In addition, all Respondents agree, by submitting a reply, that any other state or school district in the United 
States may purchase the services provided in the resulting contract, upon the same terms and conditions, and 
price, as contained in the contract, or as permitted under the laws of that jurisdiction. 

3.2 BACKGROUND 

The Florida Comprehensive Assessment Test® 2.0 (FCAT 2.0) and Florida End-of-Course (EOC) 
Assessments implement the requirements of Section 1008.22, Florida Statutes (F.S.), requiring assessments 
of the academic achievement of Florida's public school students. The FCAT 2.0/EOC Assessment program is 
composed of state-developed criterion-referenced assessments. FCAT 2.0 Reading and Mathematics have 
been administered each spring since 2011 to assess students in selected grade levels. FCAT 2.0 Science 
was introduced in spring 2012, FCAT 2.0 Writing was introduced in spring 2013, and the Florida EOC 
Assessments have been phased in beginning in 2011. Prior to FCAT 2.0, FCAT assessments were 
administered each spring since 1998, and included Reading, Mathematics, Writing, and Science. The final 
administration of FCAT 2.0 Writing, FCAT 2.0 Reading, and FCAT 2.0 Mathematics will be in spring 2014. 
The final administration of the FCAT 2.0 Reading Retake for students with this graduation requirement will be 
in spring 2016, and the Florida Algebra 1 and Geometry EOC Assessments will be eliminated after the 
summer 2017 administration. (Administration of the FCAT 2.0 Reading Retake and the Florida EOC 
Assessments will be handled in another procurement.) 

The Sunshine State Standards, which specify challenging expectations for the educational achievement of 
Florida students, were revised and approved by the State Board of Education as the Next Generation 
Sunshine State Standards (NGSSS). The Reading/Language Arts and Mathematics revised standards were 
approved by the State Board in 2007. Science and Social Studies Standards were approved in February 
2008. The NGSSS and associated course descriptions for the Florida End-of-Course Assessments are 
available at www.flstandards.org . In 2010, Florida adopted the Common Core State Standards for English 
Language Arts/Literacy and for Mathematics. As of the release of this ITN, these standards were scheduled 
to be reviewed further by Florida's stakeholders. The degree to which the standards, and by extension the 
assessments, will be adjusted is to be determined. Florida plans for the Florida Standards Assessments to be 
operational beginning in 2014-15. 

3.3 DEFINITIONS 

After the award, said Respondent will be referred to as the " Contractor ". For the purpose of this document, 
the term " Respondent " means a potential Contractor acting on its own behalf and on behalf of those 
individuals, partnerships, firms, or corporations comprising the Respondent's team. The term " Reply " means 
the complete response of the Respondent to the ITN, including properly completed forms and supporting 
documentation. The term " contract " refers to the agreement between the Department and the Contractor 
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resulting from this ITN. "Best value " means the highest overall value to the state based on objective factors 
that include, but are not limited to, price, quality, design, and workmanship. A " responsive bid" is a Reply 
submitted by a responsive and responsible vendor which conforms in all material respects to the solicitation. 
However, for purposes of determining what is material, please refer to Section 1 of this ITN. Use of the terms 
"shall," "will," and "must," or the designation of items as "requirements," does not mandate that such items are 
material. The Department reserves the right to waive deviations, in its discretion, and resolve any issues in 
the negotiation phase. " Deliverable " means a tangible, specific, quantifiable and measurable event or item 
that must be produced to complete a project or part of a project directly related to the scope of services. 

Additional definitions/terms as follows: 

a. AL - Achievement Level 

b. Braille - Paper-based accommodation provided for eligible students with disabilities 

c. CBT - Computer-based test; computer-based testing 

d. Cohort - For the purposes of this ITN, refers to a group of students that enters grade 9 and is enrolled at 
the high school for a total of four years 

e. District Assessment Coordinator - Primary contact(s) in each school district with whom FDOE 
communicates regarding the administration of K-12 statewide assessments and who oversees the 
administration of statewide assessments at all participating schools in that district 

f. DAR - District Aggregated Results files used for reporting purposes 

g. DIF - Differential Item Functioning 

h. DSR - District Student Results file used for reporting purposes 

i. DSS - Developmental Scale Score 

j. EOC- End-of-Course - Refers to an assessment thatis associated with a specific course 

k. FDOE (DOE) - Florida Department of Education 

I. FPO - Placeholder for a graphic used in a publication 

m. FSA - Florida Standards Assessment, the current name of the assessments to be administered as part of 
this ITN 

n. IEP - Individual Educational Plan forstudents with disabilities (ESE) 

o. IRT - Item Response Theory 

p. Large Print - Paper-based or computer-based accommodation provided for eligible students with 
disabilities; large print usually refers to at least 18 point font 

q. MS (Manuscript) - The initial production round when a document is composed and first reviewed 

r. MSID (Master School Identification File) - A file maintained by the Department and available online used 
to identify all active schools that will participate in statewide assessment administrations 

s. One-ltem-Per-Page - Paper-based accommodation provided for eligible students with disabilities; only 
one item appears on each printed page 

t. P&D - Packaging and distribution; often used in reference to the specifications that describe a 
contractor's processes of packaging and distributing all statewide assessment materials prior to each 
assessment administration 

u. PBT - Paper-based test; paper-based testing 

v. P relD (P reidentification) - P rocess for identifying students prior to an administration to print labels with all 
of the student's information (PrelD labels) or to create computer-based student records that will be used 
for testing 

w. PT (Program team) - Contractor staff members with program-wide expertise who are expected to assist 
in developing accurate instructions and providing specific information related to the contractor's 
processes and procedures regarding Florida's statewide assessments 
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x. Rangefinding - Process required for scoring purposes by which student responses to performance tasks 
are used to illustrate score points on a rubric 

y. RD - Results Delivery (RD1, RD2, etc.) 

z. SAR - State Aggregated Results files used for reporting purposes 

aa. School Assessment Coordinator - A designated school-level assessment contact who oversees 
administration of statewide assessments at that school, training of test administrators and proctors, and 
preparing and returning test materials 

bb. Seals - Method of maintaining security of test content until a test session begins; paper-based tests may 
have one or more paper or plastic seals that students can easily break without tearing the test book, and 
computer-based tests may have an electronic means of sealing a test session (e.g., content is locked until 
a specific code is entered that allows access) 

cc. Section 504 Plan - Plan detailing accommodations for students with temporary disabilities (may not be 
ESE) 

dd. Section 508 Compliant (ADA Compliant) - Section 508 requires that electronic and information 
technologies which are procured or developed, be maintained by federal agencies and be accessible to 
people with disabilities, unless it poses an undue burden to do so. Accessibility standards needed to 
comply with this law were developed by the United States Architectural and Transportation Barriers 
Compliance Board. Section 508 of the Rehabilitation Act of 1973 is in the federal code of regulations (29 
U.S.C. § 794(d). 

ee. SES (Scaling, Equating, and Scoring) - The set of processes used to ensure that assessment results are 
valid, reliable, and accurate 

ff. SS - Scale score 

gg. SSR - State Student Results files used to check data for reporting purposes 
hh. STMs - Sample Test Materials 

ii. Test Administrator - A certified educator who administers statewide assessments to students in a testing 
room; responsible for reading administration scripts to students and maintaining a secure testing 
environment 

jj. TAC (Technical Advisory Committee) - A panel of experts who review and respond to proposed 
modifications and processes for the assessment program 

kk. TAM - Test administration manual 

II. TC (Technology Coordinator) - A designated coordinator at the school or district level responsible for 
preparing for computer-based testing and addressing technical issues during testing 

mm.TDC - Test Development Center, which is overseen by the FDOE and staffed with editors and Florida 
educators; TDC provides quality assurance for all test content and related products, whether customized 
or derived from existing products 

nn. TM - Test Materials Shipment (TM1, TM2, etc) 

3.4 CRITICAL EVENT DATES 

These are tentative dates for critical events, and are subject to change. 

• October 25, 2013 - ITN Released 

• November8, 2013 @ 2:00 PM, ET - Deadline for Technical Questions 

• November 15, 2013 - Department Response to Technical Questions received by deadline 

• December 12, 2013 @ 2:00 PM, ET - Technical & Price Replies are due 
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• December 12, 2013 @ 2:30 PM - Technical Replies will be opened in Room 332 Turlington Building 
(Step 1) 

• J anuary 10, 2014 - The Department will evaluate the Technical Replies in Room 1706 Turlington Building 
@ 9:00 AM (Step 2) 

Price Replies will be evaluated by the Bureau of Contracts, Grants and Procurement Management Services 
staff during the evaluation stage. 

• J anuary 28, 2014 - The Negotiation Committee will start negotiations on or about (Step 4) 

• February 13, 2014 BAFO's due on or about 

• February 25, 2014 - ITA Meeting on or about 

The Intent to Award will be posted for 72 hours upon completion of the negotiation meeting and a final 
decision by the Department. 



SECTION 4 - SPECIAL INSTRUCTIONS 

4.0 PRE-SOLICITATION CONFERENCE: 

A P re-Solicitation Conference will not be held. 

4.1 SITE INSPECTION: 

A Site Inspection will not be held. 

4.2 VISITOR'S PASS TO THE TURLINGTON BUILDING 

Each visitor to the Turlington Building is required to sign in and obtain a Visitor's Pass at the security desk in 
the main lobby. Please allow at least 15 minutes prior to Reply due time if hand-delivering the Reply to the 
Bureau of Contracts, Grants and Procurement Management Services. 

4.3 REPLY QUESTIONS & ANSWERS 

Any technical questions arising from this ITN must be forwarded, in writing, to the purchasing agent identified 
below. The Department's written response to written inquiries submitted timely by Respondents will be posted 
on the Florida Vendor Bid System (VBS) at www.myflorida.com (click on Business, then click on Doing 
Business with the state, under Everything for Vendors and Customers, click on the Vendor Bid System, then 
Search Advertisement; select the Department of Education in the Agency drop down window and initiate 
search), under this Reply number. It is the responsibility of all potential Respondents to monitor this site for 
any changing information prior to submitting a Reply. 

Only timely received written inquiries will be "officially" addressed by the Department. 

WRITTEN QUESTIONS should be submitted to: 

Bureau of Contracts, Grants and Procurement Management Services, Fran Shewan, 325 West Gaines 
Street, 332 Turlington Building, Tallahassee, Florida 32399-0400, fshewan@ fldoe.org (email preferred), 
or via Fax Number: (850) 245-0719. See SECTION 3.4 Critical Event Dates for question due dates. 

4.4 PROCUREMENT PROTESTS / NOTICE OF RIGHTS 

Pursuant to F.S., Section 120.57(3) (b): 

Any person who is adversely affected by the agency decision or intended decision shall file with the agency a 
notice of protest in writing within 72 hours after the posting of the notice of decision or intended decision. With 
respect to a protest of the terms, conditions, and specifications contained in a solicitation, including any 
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provisions governing the methods for ranking bids, proposals, or replies, awarding contracts, reserving rights 
of further negotiation, or modifying or amending any contract, the notice of protest shall be filed in writing 
within 72 hours after the posting of the solicitation. The formal written protest shall be filed within 10 days after 
the date the notice of protest is filed. Failure to file a notice of protest or failure to file a formal written protest 
shall constitute a waiver of proceedings under this chapter. The formal written protest shall state with 
particularity the facts and law upon which the protest is based. Saturdays, Sundays, and state holidays shall 
be excluded in the computation of the 72-hour time periods provided by this paragraph. 

Section 120.57(3) (a) provides: 

Failure to file a protestwithin the time prescribed in Section 120.57(3), F.S. or failure to postthe bond or other 
security required by law within the time allowed for filing a bond shall constitute a waiver of proceedings under 
Chapter 120, F.S." 

Florida Administrative Code (F.A.C.) Rule 28-110.002(2) defines the term "decision or intended decision," and 
includes the solicitation terms (and any addenda), the award of the contract, and a rejection of all bids. 

At the time of filing the Formal Written Protest the protestor must also file a Protest Bond payable to the 
Department in an amount equal to one percent (1%) of the estimated contract amount. F.S., Section 
287.042(2) (c) and F.A.C. Rule 28-110.005 contain further terms relating to the Protest Bond, including how to 
determine the estimated contract amount. In lieu of a Protest Bond, the Department will accept cashier's 
checks, official bank checks or money orders. The bond shall be conditioned upon the payment of all costs 
and charges that are adjudged against the protestor in the administrative hearing in which the action is 
brought and in any subsequent appellate court proceeding. 

The Notice of Protest, Formal Written Protest, and Protest Bond shall be filed with the issuing office as defined 
in SECTION 4.3 above. 

4.5 ORAL INSTRUCTIONS /CHANGES TO THE INVITATION TO NEGOTIATE (ADDENDA) 

No negotiations, decisions, or actions will be initiated or executed by a Respondent as a result of any oral 
discussions with a state employee. Only those communications which are in writing from the Bureau of 
Contracts, Grants and Procurement Management Services, will be considered as a duly authorized 
expression on behalf of the Department until negotiation is awarded. 

Notices of changes (addenda) will be posted on the VBS, under this Reply number. It is the responsibility of all 
potential Respondents to monitor this site for any changing information prior to submitting a Reply. All 
addenda should be acknowledged by the Respondent's signature and subsequent submission of addenda 
with Reply when so stated in the addenda. 

4.6 MODIFICATIONS, RESUBMITTAL AND WITHDRAWAL 

Respondents may modify submitted replies at any time prior to the Reply due date. Requests for modification 
of a submitted Reply shall be in writing and should be signed by an authorized representative of the 
Respondent. Upon receipt and acceptance of such a request, the entire Reply will be returned to the 
Respondent and not considered unless resubmitted by the due date and time. Respondents may also send a 
change in a sealed envelope to be opened at the same time as the Reply. The ITN number, opening date and 
time should appear on the envelope of the modified Reply. 

Unless specifically requested by the Department, any amendments, revisions, or alterations to Replies will not 
be accepted after the closing for the receipt of Replies. 

4.7 RESTRICTIONS ON COMMUNICATIONS WITH DEPARTMENT STAFF 

Respondents shall not communicate with any Department staff concerning this ITN except for the Department 
contact person identified in SECTION 4.3 REPLY QUESTIONS & ANSWERS of this ITN. Only those 
communications which are in writing from the Bureau of Contracts, Grants, and Procurement Management 
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Services shall be considered as a duly authorized response on behalf of the Department. For violation of this 
provision, the Department reserves the right to reject a Respondent's Reply. 

Respondents to this solicitation or persons acting on their behalf may not contact, between the release of the 
solicitation and the end of the 72-hour period following the agency posting the notice of intended award, 
excluding Saturdays, Sundays, and state holidays, any employee or officer of the executive or legislative 
branch concerning any aspect of this solicitation, except in writing to the procurement officer or as provided in 
the solicitation documents. Violation of this provision may be grounds for rejecting a Reply. 

4.8 CONFIDENTIAL, PROPRIETARY, OR TRADE SECRET MATERIAL 

The Department takes its public records responsibilities as provided under Chapter 119, F.S., and Article I, 
Section 24 of the Florida Constitution, very seriously. If Respondent considers any portion of the 
documents, data or records submitted in response to this solicitation to be confidential, trade secret 
or otherwise not subject to disclosure pursuant to Chapter 119, F.S., the Florida Constitution or other 
authority, Respondent must clearly mark and identify in its Reply those portions which are 
confidential, trade secret or otherwise exempt. Respondent must also simultaneously provide the 
Department with a separate redacted copy of its Reply. This redacted copy shall contain the Department's 
solicitation name, number, and the name of the Respondent on the cover, and shall be clearly titled "Redacted 
Copy." The Redacted Copy shall be provided to the Department at the same time Respondent submits 
its Reply to the solicitation and must only exclude or obliterate those exact portions which are claimed 
confidential, proprietary, or trade secret. The Respondent should also provide two (2) electronic 
copies (compactdisc (CD), flash drive, etc.) of their Redacted Copy. 

Respondent shall be responsible for defending its determination that the redacted portions of its Reply are 
confidential, trade secret or otherwise not subject to disclosure. Further, Respondent shall protect, defend, 
and indemnify the Department for any and all claims arising from or relating to Respondents determination 
that the redacted portions of its Reply are confidential, proprietary, trade secret or otherwise not subject to 
disclosure. 

If Respondent fails to submit a Redacted Copy with its Reply, the Department is authorized to produce the 
entire documents, data or records submitted by Respondent in answer to a public records request for these 
records. 

4.9. POOR PERFORMANCE NOTICE 

The Respondent should provide for both the Respondent and its employees, subcontractors, and 
subcontractor employees, copies of any and all documents regarding complaints filed, investigations made, 
warning letters or inspection reports issued, any notice of breach, notice of default, termination notice, 
suspension notice, or any disciplinary action initiated or taken under any contract or job performance within 
the past seven (7) years. For each instance listed, provide a narrative summary of the contract's purpose and 
scope of work, the Respondent's performance, including the concerns of the project owner, and any major 
adverse findings. In addition, provide the contract or job number, the name of the owner, the term of the 
contract, the name, address, and telephone number of the owner's contract manager. Please also include 
any relevant documentation evidencing the performance issues. 

The Department reserves the right to seek further information on this matter from the Respondent or to make 
inquiries with the project owner. The information obtained from this review may be reflected in the 
Respondent's score or used to declare the Respondent not a responsible vendor. 

4.10 WITHDRAWAL OF A REPLY 

A Respondent may withdraw a Reply by written notice to the Department on or before the deadline specified 
for the receipt of Replies in SECTION 3.4 CRITICAL EVENT DATES of this ITN. Such written notice is to be 



Florida Standards Assessments, ITN 2014-47 



13 



submitted to the Issuing Office at the address specified in SECTION 4.3 REPLY QUESTIONS AND 
ANSWERS of this ITN. 

4.11 DISCLOSURE OF REPLY CONTENTS 

All documentation produced as part of this solicitation shall become the exclusive property of the state and 
may not be removed by the Respondent or its agents. All Replies shall become the property of the state and 
shall not be returned to Respondent. The Department shall have the right to use any or all ideas or 
adaptations of the ideas presented in any Reply. Selection or rejection of a Reply shall not affect this right. 

4.12 AWARD 

As in the best interest of the State, the right is reserved to award based on all or none, groups of services, 
or any combination thereof, to a responsive, responsible Respondent. As in the best interest of the state, 
the right is reserved to reject any and/or all Replies or to waive any minor irregularity in replies received. In 
addition, the Department reserves the right, in its discretion, to correct deviations during the negotiation phase. 
Conditions which may cause rejection of Replies include, without limitation, evidence of collusion among 
Respondents, obvious lack of experience or expertise to perform the required work, failure to perform, or meet 
financial obligations on previous contracts. 



5.0 AUTHORIZED TO DO BUSINESS IN THE STATE OF FLORIDA 

Foreign corporations and foreign limited partnerships should be authorized to do business in the state of 
Florida. Domestic corporations should be active and in good standing in the State of Florida. Such 
authorization and status should be obtained by the Reply due date and time, but in any case, must be 
obtained prior to contract execution. For authorization, contact: 

Florida Department of State 
Tallahassee, Florida 32399 
(850) 245-6053 

5.1 LICENSED TO CONDUCT SERVICES IN THE STATE OF FLORIDA 

If the services being provided require that individuals be licensed by the Florida Department of Business and 
Professional Regulation or any other state or federal agency, such licenses should be obtained by the Reply 
due date and time, but in any case, must be obtained prior contract execution. For state licensing, contact: 

Florida Department of Business and Professional Regulation 
Tallahassee, Florida 32399-0797 
(850) 487-9501 

5.2 OTHER CONDITIONS 

Other conditions which may cause rejection of replies include, without limitation, evidence of collusion among 
Respondents, obvious lack of experience or expertise to perform the required work, failure to perform or meet 
financial obligations on previous contracts. 

5.3 IDENTICAL EVALUATION OF REPLIES 

Whenever two (2) or more replies which are equal with respect to price, quality, and service are received, the 
Department will determine the order of award using the criteria established in 60A-1.011, F.A.C. The "Drug- 
Free Workplace Program Certification" can be found as Attachment 2. 
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5.4 DISCLOSURE STATEMENT 

The Disclosure Statement Form (Attachment 4) should be signed and submitted with the Reply. 

5.5 SUB-CONTRACTING 

This contract or any portion thereof shall not be sub-contracted except as permitted herein, with the prior 
written approval of the Department. No sub-contract shall, under any circumstances, relieve the Contractor of 
its liability and obligation under this contract; and despite any such sub-contracting the Department shall deal 
through the Contractor, which shall retain the legal responsibility for performing the Contractor obligations. 

The Prime Contractor shall report all Minority Subcontractors, identifying the Name, Address, Type of 
Certification and Dollar Amount on the Utilization Summary form, attached as Attachment 5. A list of 
subcontractors shall be provided to the Department's contract manager upon execution of the Contract. The 
Prime Contractor shall submit the Utilization Summary form with each invoice submitted for payment. The 
form must be submitted with all invoices, regardless if funds have not been spent with a Minority 
Subcontractor for the period covered by the invoice. The Office of Supplier Diversity, Florida Department of 
Management Services, will assist in furnishing names of qualified minorities. The Office of Supplier Diversity 
can be reached at (850) 487-0915; the Internet Web address is 
http://dms.myflorida.com/other programs/office of supplier diversity osd/ . 

5.6 CONTRACTUAL OBLIGATIONS 

The Department's Contract Standard Terms and Conditions are incorporated in this ITN as Attachment 6 and 
will govern the relationship between the Department and the Contractor. A Reply submitted by the successful 
Respondent(s) shall be incorporated into the final contract(s). The terms and conditions of this ITN shall 
control notwithstanding any statement to the contrary by the Respondent, unless such terms and 
conditions are modified by the Department during the negotiation phase. 

5.7 METHOD OF PAYMENT 

Compensation and payment will be made in accordance with the terms and conditions of the contract. 

5.8 CONVICTED VENDOR LIST 

A company placed on the Convicted Vendor List may not submit a Reply or be awarded a contract to provide 
any goods orservices pursuant to Rule 60A-1.006 F.A.C. The "Convicted Vendor List" is published at: 

http://www.dms.myflorida.com/business operations/state purchasing/vendor information/convicted suspen 
ded discriminatory complaints vendor lists/convicted vendor list . 

5.9 DIVERSITY IN CONTRACTING 

The State of Florida is committed to supporting its diverse business industry and population through ensuring 
participation by minority-, women-, and service-disabled veteran business enterprises in the economic life of 
the state. The state of Florida Mentor Protege Program connects minority-, women-, and service-disabled 
veteran business enterprises with private corporations for business development mentoring. We strongly 
encourage firms doing business with the State of Florida to consider this initiative. For more information on 
the Mentor Protege Program, please contact the Office of Supplier Diversity at (850) 487-0915. 

The State is dedicated to fostering the continued development and economic growth of small, minority-, 
women-, and service-disabled veteran business enterprises. Participation by a diverse group of Vendors 
doing business with the state is central to this effort. To this end, it is vital that small, minority-, women-, and 
service-disabled veteran business enterprises participate in the State's procurement process as both 
Contractors and sub- contractors in this solicitation. Small, minority-, women-, and service-disabled veteran 
business enterprises are strongly encouraged to contribute to this solicitation. 
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Information on Certified Minority Business Enterprises (CMBE) and Certified Service-Disabled Veteran 
Business Enterprises (C5DVBE) is available from the Office of Supplier Diversity at: 

http://dms.myflorida.com/other programs/office of supplier diversity osd/ . 

5.10 COSTS INCURRED IN RESPONDING 

This ITN does not commit the Department or any other public agency to pay any costs incurred by the 
Respondent in the submission of a Reply or to make necessary studies or designs for the preparation thereof, 
nor to procure or contract for any articles or services. 

5.11 SUBMISSION OF REPLIES BY SUBSIDIARIES OR AFFILIATES 

A Respondent, its subsidiaries, affiliates, or related entities is limited to one (1) Reply. Submission of more 
than one (1) Reply per activity by a Respondent may cause the rejection of all Replies submitted by the 
Respondent. In the alternative, the Department my decide, in its sole discretion, which Reply to 
evaluate and consider. A subsidiary or affiliate of a prime Respondent may also be included as a 
subcontractor in another Respondent's Reply. 

5.12 PROHIBITION OF GRATUITIES 

By submission of a Reply, the Respondent certifies that no elected or appointed official or employee of the 
state of Florida has or will benefit financially or materially from this procurement. Any contract arising from this 
procurement may be terminated by the Department if it is determined that gratuities of any kind were either 
offered to or received by any of the aforementioned officials or employees from the Respondent or its agents 
or employees. 

5.13 INDEPENDENT PRICE DETERMINATION 

A Respondent shall not collude, consult, communicate, or agree with any other Respondent regarding this 
procurement as to any matter relating to the Respondent's Reply. 

5.14 FINANCIAL CONSEQUENCES 

The Contractor shall submit to the Department for payment the above deliverables and those proposed and 
accepted by the Department that are satisfactorily completed in accordance with the terms and conditions set 
forth in SECTION 7.0 SCOPE OF SERVICES and SECTION XXXXX DELIVERABLES. Payment shall be 
delayed for any deliverables that are not satisfactorily completed until all deficiencies are corrected and the 
services have been accepted by the Department. In addition, delays in providing completed deliverables will 
subject the Contractor to damages. 

5.15 PERFORMANCE BOND 

The Contractor shall supply to the Department a Performance Bond in the amount of ten percent (10%) of the 
amount of the annual award. The surety shall be in a form acceptable to the Florida Department of Education, 
such as a bond, cashier's check, certified check or money order. A Surety must be authorized to do business 
in the state of Florida. The Performance Bond shall be executed and furnished to the Department within ten 
(10) calendar days prior to the Contractor beginning work under the contract. The performance bond must 
state that it includes coverage of liquidated damages assessed against the contractor. A performance bond is 
not required from Florida State Universities. 

Cap: For each calendar, contract or fiscal year which corresponds to the period of the performance bond 
specified, the cumulative total of liquidated damages for which the Contractor(s) may be liable shall not 
exceed, annually, ten percent (10%) of the total contract amount at the time of assessment of liquidated 
damages. 
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5.16 PARTICIPATION IN FUTURE STAGES OF THIS PROj ECT 

As stated in Chapter 287.057 F.S. 

17(a)c A person who receives a contract that has not been procured pursuant to subsections (1) through (3) 
to perform a feasibility study of the potential implementation of a subsequent contract, who 
participates in the drafting of a solicitation or who develops a program for future implementation, is 
not eligible to contract with the agency for any other contracts dealing with that specific subject 
matter, and any firm in which such person has any interest is not eligible to receive such contract. 
However, this prohibition does not prevent a vendor who responds to a request for information from 
being eligible to contract with an agency. 

5.17 ACCESSIBLE ELECTRONIC INFORMATION TECHNOLOGY 

Respondents submitting Replies to this solicitation must provide electronic and information technology resources 
in complete compliance with the accessibility standards required by Section 282.601-282.606, F.S., and Rule 60- 
8.002, F.A.C. These standards establish a minimum level of accessibility. 

5.18 SCRUTINIZED COMPANIES LISTS (as applicable) 

Section 287.135, Florida Statutes, requires that at the time a company submits a bid or proposal for a contract 
for goods or services of $1 million or more, the company must certify that the company is not on either the 
Scrutinized Companies with Activities in Sudan List or the Scrutinized Companies with Activities in the Iran 
Petroleum Energy Sector List. Both lists are created pursuantto section 215.473, Florida Statutes. 

Replies of $1 million or more should include the attached Scrutinized Companies Lists Form (Attachment 6) 
to certify the respondent is not on either of those lists. The Form should be submitted with the Technical 
Reply. 

5.19 LIQUIDATED DAMAGES 

Accurate and timely delivery is imperative and, as a result, the contract will include the following provisions for 
the completion of the Work Tasks/Activities and Interim Deliverables and Final Deliverables identified in this 
ITN. 

The Contractor(s)'s failure to complete Work Tasks/Activities and Interim Deliverables and Final Deliverables 
both correctly and on time will result in substantial injury to the Department and the State of Florida, but the 
amount of damages resulting from such injury cannot be calculated with certainty. Each such failure to 
complete Work Tasks/Activities and Interim Deliverables and Final Deliverables both correctly and on time is 
hereinafter referred to as a default. Defaults shall be deemed corrected on the date that the Work 
Tasks/Activities and Interim Deliverables and Final Deliverables have been correctly completed. For each 
default, the Contractor(s) shall be liable to the Department for liquidated damages and not as a penalty, as set 
forth below: 

a. Tasks/Activities and Interim Deliverables: For each default on performance and completion of Work 
Tasks/Activities and Interim Deliverables (see SECTION 7.8), it is agreed that for each and every day 
work remains uncompleted beyond the time set for its completion the Contractor(s) shall be liable to the 
Department in a specified amount of three thousand dollars ($5,000.00) per business day as liquidated 
damages for such delay, and not as a penalty, from the date of notice by the Department until the default 
has been corrected. 
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b. Final Deliverables: For each default on completion of Final Deliverables (see SECTION 7.8), the 
Contractor(s) shall be liable to the Department in a specified amount per calendar day from the date of 
notice by the Department until the default has been corrected. The specified amount of damages is 
defined below: 

Schedule of Liquidated Damages 





Number of business days 
elapsed after the default 

First 

Second 

Third 

Fourth 

Fifth 

Sixth 

Seventh/ subsequent days 



Liquidated Damages 



$5,000.00 

$15,000.00 

$30,000.00 

$45,000.00 

$60,000.00 

$75,000.00 

$125,000.00 per day 



c. Caj): For each calendar, contract, or fiscal year which corresponds to the period of the performance bond 
specified in ITN Section 5.15, the cumulative total of liquidated damages for which the contractor may be 
liable shall not exceed, annually, ten percent (10%) of the total contract amount at the time of assessment 
of liquidated damages. 

d. Collection: At its option, the Department shall collect liquidated damages by making claims against the 
performance bond, from time to time, until the bond has been exhausted or by deducting the liquidated 
damages from contract payments to the contractor. In the alternative, the Contractor may make payment 
to the Department in the amount of such liquidated damages. 

e. Deadline Adjustments: 

(1) In the event the contractor wishes to propose a change of deadline for a Critical Work Task, the 
contractor may propose the change to the Department, in writing. The Department has no obligation 
to agree to the change, but shall notify the contractor, in writing, as to whether or not it agrees to the 
proposed change. If the Department agrees, the modified deadline shall become the effective 
deadline for purposes of assessing liquidated damages. 

(2) In the event the Department wishes to change a deadline for an Extremely Critical Work Task to a 
date which shortens the contractor's time for meeting the deadline, it shall notify the contractor of the 
new deadline in writing in the form of a contract amendment. If the contractor determines that by 
exercise of every reasonable effort, the contractor will be unable to meet the new deadline, the 
contractor shall offer a proposed shortened deadline which the contractor can meet by the exercise of 
every reasonable effort. 

(3) In the alternative, the contractor may propose a contract amendment in which the additional 
personnel hours needed to achieve the shortened deadline are compensated by a price adjustment. 

(4) If the parties are unable to reach agreement, the Department shall determine the deadline. However, 
the issue of whether liquidated damages apply to the contractor's failure to meet the shortened 
deadline shall be reached as follows. The parties shall submit to binding arbitration the issue of what 
date (earlier than the original deadline) the contractor could have completed the Extremely Critical 
Work Task on time and correctly by exercising every reasonable effort. The date determined through 
arbitration shall be the deadline for purposes of assessing liquidated damages. Arbitration shall be 
conducted pursuant to the procedures of the American Arbitration Association, subject to the Florida 
Arbitration Code, Chapter 682, F.S. 
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f. Waiver: The Department, in its sole discretion, may waive the imposition of liquidated damages or a 
portion thereof in a given instance. Such waiver, in any instance, shall not constitute a waiver in any 
future instance, nor establish any right on behalf of the contractor to a waiver. 

g. Department Approval: In situations where in the contractor must obtain the Department's approval of an 
activity or product before the contractor can complete a Critical Work Task, the contractor shall be 
responsible for providing a reasonable time for the Department to complete its review and for the 
contractor to correct any deficiencies. In the event the Department's review is not conducted within a 
reasonable time, as described in Section 7.7.4, that shall be grounds for the contractor to propose a 
deadline adjustment pursuant to Subsection 5.19(e)(1), above. The parties' contract managers may 
agree, in writing, in advance, as to the reasonable time for the Department's review of a specific activity or 
product. 

SECTION 6 - SPECIAL INSTRUCTIONS - REPLY FORMAT & CONTENT 

6.0 REPLY SUBMISSION 

Both Technical and Price Replies will be received as specified in SECTION 3.4 CRITICAL EVENT DATES . 

All Replies and associated forms should be signed and dated in ink by a duly authorized representative of the 
Respondent. The overall Reply should be written in a concise manner, which is conducive to effective 
evaluation and selection. 

All Replies and related documents submitted in response to this ITN shall become the property of the state. 

6.1 MAIL OR DELIVER REPLIES TO: (Do Not Fax or E-Mail) 

Florida Department of Education 

Bureau of Contracts, Grants and Procurement Management Services 
Attn: Fran Shewan 
325 West Gaines Street 
332 Turlington Building 
Tallahassee, Florida 32399-0400 

6.2 REPLY FORMAT INSTRUCTIONS 

This section contains instructions that describe the expected format for the Reply. All Replies submitted 
should contain two parts and be marked as follows: 

PARTI TECHNICAL REPLY NUMBER ITN 2014-47 
( One Separately Sealed Package for Technical) 

PART II PRICE REPLY NUMBER ITN 2014-47 
( One Separately Sealed Package for Prices) 

THE SEPARATELY SEALED PACKAGES MAY BE MAILED TOGETHER IN ONE ENVELOPE OR BOX. 

6.2.1 Preliminary Administrative Review 

The absence of any of these documents may result in a determination that the Reply is non- 
responsive and the Reply shall not be evaluated. The Reply forms furnished must be used when 
submitting the Reply. Forms are to be filled out in ink or typewritten. Administrative Review information 
shall be included in the Technical Reply portion (Part I) of the Reply and consist of the following: 

■ All Addenda 

■ DUNS Number 
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■ Disclosure Statement Form (Attachments) 

■ Reference Form (Attachment 4) - Provide at least three (3) references, which demonstrate efforts 
comparable to the one described in this ITN. The Department reserves the right to contact the references 
regarding the services provided. Any information provided will be subject to the requirements of the 
Florida Public Records Law. 

■ Transmittal Letter - Provide a Transmittal Letter (on Company Letterhead) that contains the following: 

a statement confirming that the person signing the Reply is authorized to represent the Respondent 
and bind the Respondent relative to all matters contained in the Respondent's Reply 

■ the company's federal tax identification number 

■ the company's DUNS number 

a statement confirming that the Respondent has read, understands, comply and agrees to all 
provisions of this ITN, unless changed during the negotiation process. By submitting a Reply, 
the Respondent agrees to the terms and conditions of the ITN notwithstanding any statement 
in the Reply to the contrary. However, the Department will attempt to resolve any 
disagreements in the negotiation sessions. 

a statement confirming that the Respondent is authorized to conduct business in Florida in 
accordance with the provisions of Chapter 607, F.S. In lieu of such statement, the Respondent 
alternatively should confirm that authorization to do business in Florida will be secured prior to the 
award of the contract. 

■ a statement confirming that the Respondent is registered on the MyF loridaMarketP lace website in 
accordance with the provisions by the State of Florida. In lieu of such statement, the Respondent 
should alternatively confirm that registration authorization will be completed prior to the award of the 
contract. 

■ a statement confirming that the Respondent has electronically registered a valid W-9 with the 
Department of Financial Services (DFS). DFS is ready to assist any vendors with questions, and 
vendors must submit their W-9 forms electronically at https://flvendor.myfloridacfo.com . Contact the 
DFS Customer Service Desk at (850) 413-5519 or FLW9(g> myfloridacfo.com with any questions. In 
lieu of such statement, the Respondent should alternatively confirm that registration will be completed 
prior to the award of the contract. 

6.2.2 Technical Reply (Parti) (20_hard copies) 
(Do not include price information in Parti) 

The Respondentshould submit one (1) original in hard copy, nineteen (19) hard copies and twenty (20) in 
electronic format (compact disc (CD), flash drive, etc.) copies in Microsoft Word 5.0 or higher, or 
Adobe Acrobat of the Reply which is to be divided into the sections described below. The Department 
expects all Replies to be in this format. Failure of the Respondent to follow this outline may result in the 
rejection of the Reply, or result in a lower evaluation score. Any rejection of a Reply, or other adverse affect 
on a Respondent's Reply, due to its failure to properly organize its Reply shall be at the sole risk of the 
Respondent. The Technical Reply should be submitted in a separate sealed package marked "TECHNICAL 
REPLY FOR ITN 2014-47". 

1. EXECUTIVE SUMMARY 

The Executive Summary should be written in non-technical language to summarize the Respondents 
overall capabilities and approaches for accomplishing the services specified herein. Evidence should 
include, but not be limited to, respondent's mission, date of founding, size, and experience. This section 
should also demonstrate the Respondent's clear understanding of the Department's goal to securely and 
reliably develop, administer, score, and report Florida's assessments in an efficient and cost-effective 
manner. The Executive Summary should be limited to no more than six (6) pages. 
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2. QUALIFICATIONS AND EXPERIENCE 

The respondent should provide a description of qualifications and prior experience performing tasks 
similar to those required in this ITN. The discussion should include a description of the respondent's 
background and relevant experience that qualifies it to provide the products and services required by this 
ITN. 

The respondent should document the contracted services for previous assessment projects similar to the 
one described in this ITN and any other projects the respondent believes will document its corporate 
capability. For each, the documentation should include a description of the services and products 
delivered; the contract period; and the name, address, and telephone number of a contact person for each 
of the contracting agencies. 

If subcontractors are to be used for any portion of the work activities, the respondent should name the 
subcontractor(s) and document the experience and qualifications of the subcontractor(s) in performing 
tasks identical or similar to those they will be asked to do. A separate chart in the proposal may be used to 
identify all of the subcontractors proposed to be involved in the project and the services they are expected 
to provide. The respondent also should provide examples of materials that demonstrate the quality of the 
work done by the respondent on similar projects. 

The proposal should include information sufficiently descriptive to permit the proposal evaluation 
committee to consider the experience and capability of the prime contractor as well as the subcontractors. 

As part of the proposal evaluation, the proposal evaluation committee will contact one or more of the 
agencies for whom the respondent previously had been engaged to obtain information and 
recommendations. Committee members may also depend upon and use their own personal knowledge of 
the respondent, the subcontractors, and the proposed personnel as they evaluate each proposal. 

3. TECHNICAL PLAN 

The technical plan should provide a description of the respondent's proposal to meet the requirements of 
this ITN. This discussion should encompass all of the requirements of Section 7.1.0 through Section 
7.6.3.6.3 and associated deliverables of this ITN and should be organized and numbered in accordance 
with the organization and numbering system used in Sections Section 7.1.0 through Section 7.6.3.6.3. 
The technical plan should describe the methodologies, quality management standards and best practices 
that will be employed in managing and completing the requirements. The descriptions should be 
thorough, clear, yet as concise as possible. The Respondent should also identify where significant 
difficulties may be anticipated and describe any specific techniques proposed to be used to resolve the 
difficulties. 

TECHNICAL APPROACH 

The Respondent should explain the approach, capabilities, and means to be used in accomplishing the tasks 
in the Scope of Services, and where significant development difficulties may be anticipated and resolved. Any 
specific techniques to be used should also be addressed. 

TECHNICAL APPROACH ORGANIZATION OF THE INVITATION TO NEGOTIATE 

Section 7.1 of the ITN states the purpose of the ITN; describes the current Florida statewide assessment 
program and modifications to the program that will be implemented under the ITN; and provides information 
for preparing replies. 

Sections 7.2 through 7.7 describe the scope of work for the years covered by this ITN. References are 
made in Sections 7.2 through 7.7 to the appendices, which provide detailed information about printed 
products, reports, shipments, and other elements of Florida's assessment program. Each section includes a 
description of products and services to be provided. 
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Section 7.2.0: Program Descriptions. An overview of the standards-based assessments requiring services 
described in this ITN. 

Section 7.3.0: Test Development. A description of the test development tasks from item writing through the 
production of final test forms. 

Section 7.4.0: Test Administration. A description of the test administration tasks, including the delivery and 
return of test materials, the production of test administration ancillary materials, providing customer service 
and technical support, needs for the online management system, and administering computer-based 
assessments. 

Section 7.5.0: Scanning, Scoring, and Reporting. A description of the scanning, scoring, and reporting 
tasks from preparing school and student information through the reporting of results. The psychometric 
aspects of scoring are covered in Section 7.6.0. 

Section 7.6.0: Scaling, Equating, Scoring, and Special Psychometric Studies. A description of the 
scaling, equating, scoring, and special psychometric studies and tasks. 

Section 7.7.0: Program Management. A description of how the work of the contract should be managed, 
including meetings, staff, and reimbursable funding categories. 

SYSTEM DEMONSTRATION (ELECTRONIC DEMONSTRATION) 

The Respondent should submit two (2) copies with the Technical Reply, in electronic format (compact 
disc (CD), DVD, flash drive, etc.) a demonstration (demo) of the proposed system. The system demo 
provided should not exceed 25 slides or 30 minutes of presentation. 

Do not include pricing information with the Technical Reply or the demo. Any inclusion of prices/discounts in 
the Technical Reply or demo may be grounds for deeming the Respondent's Reply nonresponsive. 

After the Evaluation Phase is completed, the Department will notify the Respondents invited to participate in 
Negotiations. Only those Respondents entering into Negotiations will be allowed and required to present an 
on-site oral presentation of the proposed operational system. The Respondent's on-site presentation will be 
conducted during the Negotiation meeting. The actual date, time and duration of Negotiation 
meetings/presentations are to be determined. 

4. RESPONDENT'S MANAGEMENT PLAN 

The management plan should describe the organization and document its capacity to provide the 
products and services required by this ITN. This discussion should specifically encompass the 
requirements of Section 7.7.0 through Section 7.7.10 and associated deliverables of this ITN and should 
be organized and numbered in accordance with the organization and numbering system used in Section 
7.7.0 through Section 7.7.10. The descriptions should be complete, clear, and concise. 

The management plan should identify by name the managers and professional personnel responsible for 
this project. Charts showing the organizational structure and the incorporation of proposed project staff 
into the structure should be included, as well as the amount of time in FTE person-days that each person 
will devote to the major tasks and activities. 

The Respondent should include a description of the methodology to be used to manage the project work, 
including control of costs, assuring quality, identifying and minimizing risk, and maintaining schedules; 

Vitae should be included as an appendix to the proposal and should include relevant educational 
background and professional experience. Personnel assigned by the contractor should demonstrate 
qualifications and competencies of the position to which assigned. 

It is assumed that the contractor may use outside printers for some materials, such as preprinted forms. 
P rinters will be documented as subcontractors, and the management plan should identify the proportion of 
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materials to be printed by the contractor and by outside vendors. Procedures for quality control and 
security during printing should be described. 

6.2.3 Price Reply (Part II) (20 hard copies) 

The Respondent should submit one (1) original hard copy, nineteen (19) hard copies and twenty (20) 
electronic format (compact disc (CD), flash drive, etc.) copies in Microsoft Excel 5.0 or higher. The 

Respondent's price information should be submitted on the form provided in this ITN. Prices submitted in the 
Price Reply form, Attachment 1, will be used to evaluate the price component of the evaluation. Payment for 
services provided under the contract will be made based on the deliverable schedule submitted, Section 7.8, 
The Price Reply and deliverable schedule should be submitted together in a separate sealed package marked 
"PRICE REPLY FOR ITN 2014-47" . 

6.2.4 Presenting the Reply 

The Reply should be limited to a page size of eight and one-half by eleven inches (8%" x 11"). Type size 
should not be less than a 12 point font. The Reply shall contain a table of contents, be typed single-spaced 
and have separate parts, each clearly labeled including page numbers. The information to be addressed in 
each part is described in the above sections. The absence of information or the organization of information in 
a manner inconsistent with the terms of this ITN may result in the rejection of the Reply, or a reduced 
evaluation score. Bindings and covers will be at the Respondent's discretion; however, elaborate 
notebooks/hard back binders are discouraged. 

Unnecessarily elaborate special brochures, art work, expensive paper and expensive visual and other 
presentation aids are neither necessary nor desired. 

All Respondent materials should be packaged so that each box of materials shipped to the Department does 
not exceed 25 pounds . 



SECTION 7 - SCOPE OF SERVICES 

7.0 SCOPE OF SERVICES 
7.1.0 Introduction 

This ITN addresses four broad assessment components as described below. Respondents may submit bids 
on any individual assessment component (A, B, C or D), any group of assessment components, or to provide 
all assessment components (A, B, C and D): 

A. Computer-based and paper-based assessments in English Language Arts/ Literacy in Grades 3-8 

B. Computer-based and paper-based assessments in Mathematics in Grades 3-8 

C. Computer-based and paper-based assessments in English Language Arts/ Literacy in Grades 9-11 

D. Computer-based and paper-based End-of-Course (EOC) assessments in Algebra 1, Geometry, and 
Algebra 2 

The Department will continue to phase in computer-based assessments in Grades 3-8, transitioning 
completely to computer-based by 2017-2018 according to the preferred schedule in Appendix A. However, 
depending on the final award for the individual assessments, the Department may vary when tests will be 
administered as computer-based or paper-based. The final determination of which tests will be computer- 
based or paper-based is at the Department's discretion. 

This ITN will specify a complete package of products and services so that the FSA will be developed, 
administered, scored, and reported separately from other assessments aligned to Florida's Next Generation 
Sunshine State Standards under a contract currently held by NCS Pearson. This ITN does not include the 
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Science, Social Studies, FCAT 2.0 Reading Retake, and Florida EOC Assessments aligned to the Next 
Generation Sunshine State Standards that will be covered in a separate procurement. 

Although this ITN may result in the award of multiple contracts, the term "Contractor" is used throughout and 
refers to each contractor providing the required products and services for each grade and subject for which the 
contract is awarded. 

7.1.1 Notice to Respondents 

1. This ITN provides for implementation of Florida's standards-based assessment program. The contract will 
include development, psychometric services, administration, scoring, and reporting of Florida Standards 
Assessments in English Language Arts/Literacy and Mathematics for the test administration cycles for 
2014-2015 through 2016-17 with possible extension of services in three one-year renewal periods, 2017- 
18 through 2019-2020. Replies will be evaluated on the basis of the respondent's qualifications and 
experience and the technical and cost replies for the tasks described herein for the entire span of the 
ensuing contract. 

2. This ITN defines the requirements for implementing Florida's standards-based assessment program. The 
ITN and the selected contractor's reply, together with clarifying documents, define the work to be 
conducted under contract. These documents will be incorporated into the contract resulting from the state 
standards-based assessment program project award. Because Florida's standards-based assessment 
program is technical and complex, it is possible that a responsive reply may not totally or clearly reflect 
ITN requirements in all details. If the reply of a respondent selected as a result of the bidding process is 
inconsistent with the ITN, the requirements of the ITN will prevail; the selected contractor will be expected 
to perform all ITN requirements without an increase in cost above the proposed cost. Furthermore, by 
submitting a bid the respondent is agreeing to be bound by the terms of the ITN notwithstanding any 
language in the reply to the contrary. Any discrepancies between the terms of the ITN and the reply will 
be resolved in the negotiation process at the Department's discretion, and when it is in the state's best 
interest. 

3. The Department reserves the right to substitute or modify tasks on a generally equal basis. Substitutions 
or modifications will be made prior to the selected contractor's beginning significant efforts to complete the 
task. All design changes, substitutions, or modifications will be given to the contractor in writing. It is not 
the intent of this provision to increase the scope of work of the contract but to protect the Department as 
design changes to improve the assessment become known during the implementation of development 
and administration activities. The Department also reserves the right to amend the contract resulting from 
this bid competition to add, modify, or delete designated tasks, products, and/or services related to the 
core activities of this project. 

4. In developing the solution pursuant to this procurement, a respondent may propose to provide a custom built 
solution, a solution comprised of pre-existing, proprietary materials or products that are the intellectual 
property of the respondent or third parties, or a combination thereof. A respondent may propose that all 
intellectual property developed be owned solely by the Department, owned jointly by the Department and the 
respondent or, at a minimum, that the Department be granted a perpetual and non-exclusive license for the 
use, reproduction and modification of the solution for use in Florida. The Department reserves the right to 
request alternative intellectual property arrangements or combinations of such arrangements during 
negotiations with respondents, in order to obtain a reply that provides the best value to the state. All 
permissions or licenses required for the use of materials or products that are the intellectual property of a third 
party must be obtained by the respondent. All materials, products, and licenses must be transferred to the 
Department at the end of the contract period such that the Department can maintain and sustain the solution 
beyond the end of the contract period. 

5. The selected contractor will work closely with the Department and with groups of Florida educators to 
complete all tasks. All procedures, processes, and products used by the contractor to complete contract 
work must be approved by the Department. Throughout the contract period, the contractor will confer with 
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the Department on a continuing and consistent basis and will be involved in frequent face-to-face 
meetings with the Department, as necessary. 

6. Several reimbursable funding categories identified in Section 7.7.9 are set aside for specific functions 
during the course of the project. Expenditures in these categories will be reconciled on a quarterly basis. 

7. The dates for reporting assessment results necessitate a restricted time period to complete the work of 
scoring activities. Respondents are expected to propose efficient and innovative procedures to meet 
these required reporting dates and are expected to demonstrate in their replies that they have the 
capability and are willing to commit the human and technological resources needed to complete the 
scanning, student response file processing, scoring, and reporting tasks in the required time period. 

8. The fundamental purpose of Florida's standards-based assessment program is to provide accurate 
information on student achievement. The contractor will utilize every means required to ensure that 
information created by the project is accurate. The selected contractor will be responsible for correcting 
at the contractor's expense any errors arising from activities that are the responsibility of the contractor. 
This may involve activities such as conducting analyses to identify the cause and extent of errors; 
reprogramming or reproducing products or other materials; replacing data files; reproducing reports; and 
shipping replacement products or reports to the Department or districts using expedited shipping services. 

9. News releases or public announcements pertaining to this ITN, a contract award resulting from 
the ITN, or work completed under a contract will not be made without prior written approval from 
the Department. 

7.1.2 Project Schedule 

The deliverables schedule presents a list of major project activities and the dates for completion where these 
can be determined. Many of the deliverable dates are contingent upon finalized administration dates and will 
be determined during the negotiation process. Appendix B indicates critical and extremely critical tasks and 
provides general guidelines for the timeframes for completion, but these are also contingent on finalized 
administration dates. The dates provided in 7.2.2 are subject to change during the negotiation phase. 

Respondents should provide a detailed schedule that incorporates the activities included in this ITN for the 
entire life of the contract. The schedule will indicate essential steps for the completion of project tasks, 
initiation and completion dates, and task responsibility. Tasks and responsibilities will include an explanation 
of work effort required and full-time equivalent staff (FTE) committed to this work. The reply should also 
provide total annual FTE and identify individuals and/or staff roles that will work full-time on this 
project. Respondents should be aware that the schedules for each project year overlap. To accurately 
estimate required annual resources, schedules for preceding and succeeding years need to be considered. 

Following the contract award, the schedule will be further developed into a working document that will be 
updated weekly, indicate all task completion dates, and be revised as needed with Department approval (See 
Section 7.7.6). A draft work schedule, based on a sample provided by the Department, will be prepared by the 
selected contractor for submission to the Department one week prior to the initial planning meeting, which will 
be held no later than one month after the execution of a contract for the project. The schedule will be 
submitted in hard copy and electronic format using Department-approved spreadsheet software that permits 
sorting of the file by start or end date and major task and subtask groupings. 

7.2.0 Program Descriptions 

The selected contractor will provide the Department with a criterion-referenced achievement test, aligned to 
and measuring mastery of the Florida Standards in English Language Arts/Literacy (ELA/L) and Mathematics 
proficiency for administration to Florida public school students. For purposes of responses to this ITN, the 
Florida Standards are to be considered those adopted by the Florida State Board of Education in J uly, 2010. 
Any changes to the standards following additional Florida public review scheduled for fall 2013, and resulting 
changes to the assessment will be addressed through the amendment of the contract that may be awarded as 
a result of this ITN. 
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The primary purpose of the assessment is to provide student academic achievement and learning gains data 
to students, parents, teachers, school administrators, and school district staff. This data is to be used by 
districts to improve instruction; by students, parents, and teachers to guide learning objectives; by education 
researchers to assess data; and by the public to assess the cost benefit of the expenditure of taxpayer dollars. 
The assessment must be designed to: 

1. Assess the achievement level and annual learning gains of each student in ELA/L and Mathematics. 

2. Provide data for making decisions regarding school accountability, recognition, and improvement of 
operations and management, including schools operating for the purpose of providing educational 
services to youth in Department of J uvenile J ustice programs. 

3. Identify the educational strengths and needs of students and the readiness of students to be promoted to 
the next grade level, to graduate from high school, and to succeed in college and in careers. 

4. Assess how well educational goals and curricular standards are met at the school, district, and state level, 
and at the national level, to the extent possible. 

7.2.1 Expected number of districts, schools, and students 

The expected number of students, shipping destinations, and schools are provided in Appendix C as a basis 
for preparing replies. Respondents should calculate the quantities of materials needed to implement the test 
administrations on the basis of these estimates. If the total number of tests administered across all grades 
and test administrations exceeds the number of tests described in this appendix by more than ten percent 
(10%), the Department will execute a formal change to the contractual agreement. 

Appendix C also lists the 67 school districts and approximately 13 special districts. For the school districts and 
the special districts combined, the contractor will ship testing materials to approximately 80 separate 
destinations. In the 67 school districts and the special districts combined, approximately 4,000 separate 
administration sites (including private schools, community colleges, etc.) will be involved in the test 
administrations, and the contractor will expect that number to increase by as much as five percent (5%) 
annually, calculated based on the previous year. Some of this increase is due to legislative changes that 
increase the number of students participating in Florida's statewide assessment program. If the increase of 
administration sites is beyond the anticipated five percent (5%), the incremental cost increase will be handled 
in an amendment to the contract. 

7.2.2 Potential Test Administration Dates 

Potential projected starting dates for future assessment administrations relevant to this ITN are shown in Table 
7.2.2 below for illustrative purposes only. The respondent's reply should propose starting dates for spring 
administrations that occur as late in the school year as possible. Replies should take into account the scoring 
of constructed- and extended-response items, data verification activities, and reports of results by the required 
dates (Appendix B). 
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The FSA EOC Assessment administrations will be at the same time as the Florida EOC Assessment 
administrations. These dates are also provided, for reference, in Appendix A. 

Table 7.2.2 EXAMPLE: Potential Test Administration Dates for Florida's Standard-Based Assessments 



Table 7.2.2 EXAMPLE: Potential Test Administration Dates for Florida's Standard-Based Assessments 


Year 

I cell 


Grades 3-11 ELA/L 
Grades 3-8 Mathematics 


Grade in Fl A/l Rptakp 


Algebra 1 EOC 

fipnmphv FOP 

Algebra 2 EOC 


2014-15 


March 2-13, 2015 


N/A 


April 7-25, 2015 
J uly 13-24, 2015 


2015-16 


February 29-March 11, 2016 


October 12-23, 2015 
February 29-March 11, 2016 


beptemoer 14-25, 2015 
November 30-December 18, 2015 
April 4-22, 2016 
July 11-22, 2016 < 


2016-17 


March 6-17, 2017 


October 10-21, 2016 
March 6-17, 2017 


beptemDer n-a, zuio 
November 30-December 21, 2016 
April 3-21, 2017 
J uly 10-21, 2017 ' 


2017-18 1 


March 5-16, 2018 


October 9-20, 2017 
March 5-16, 2018 


Depiemrjcr ii-zz, zui/ 
December 4-22, 2017 
April 2-20, 2018 
J uly 16-27, 2018 


2018-19 1 


March 4-15, 2019 


October 8-19, 2018 
March 4-15, 2019 


September 10-21, 2018 
December 3-21, 2018 
April 8-26, 2019 
J uly 15-26, 2019 ! 


2019-20 1 


March 2-13, 2020 


October 7-18, 2019 
March 2-13, 2020 


September 9-20, 2019 
December 2-20, 2019 

April 6-24, 2020 

J uly 13-24, 2020 1 


-Optional Renewal Period I 



7.3.0 Test Design and Development 

All services related to test design, item development and review, item banking, and test production for the FSA 
are the responsibility of the contractor selected under this ITN. Respondents may propose a customized 
assessment, or products and services derived from existing products developed and published by the 
contractor, or developed as part of other initiatives. It is understood that by the time the contract is executed 
for the services requested in this ITN, test items may have already been developed for assessments that will 
be operational in 2014-2015. However, the Department expects to be closely involved in the development and 
review of new test items to be field tested and developed in subsequent years. To the extent possible, the 
Department also expects to be involved in any test construction activities for 2015 assessments still to be 
completed after contract execution. The Department's primary goal with respect to test design is ensuring that 
the assessment measures student mastery of the Florida Standards, and accomplishes this in the most cost- 
effective way without significantly increasing testing time beyond that of current Florida assessments. A 
sample test design is provided in Appendix D. 

The Department has established a Test Development Center (TDC), staffed with editors and Florida 
educators, to provide quality assurance for all test content and related products, whether customized or 
derived from existing products. The selected contractor will work closely with the Department and TDC staff to 
ensure that the test philosophy, design, item specifications, and product quality are aligned with Florida's 
expectations. The Department and TDC will also coordinate closely with the selected contractor to conduct 
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rangefinding meetings with educators in order to generate training materials for the scoring of performance 
tasks. While the selected contractor will be required to coordinate all meeting activities and materials, the 
Department and TDC will identify invitees and assist in facilitating all meetings. Required meetings are 
referenced throughout the ITN. See Section 7.7.7 for a summary of details about educator/citizen meetings. 
Test development procedures should include review and revision of proposed test and item specifications, 
review of test blueprints, review and evaluation of items available for use on the test (item development plan), 
field testing for needed items to complete the desired test design, and final operational test creation. All 
phases of test development shall use accepted validity, reliability and other testing principles including 
Universal Test Design. A plan for complete data review of field-tested items should be delineated. The 
Department must approve the items and the test forms before they become operational. Respondents who 
propose solutions derived from existing products that are used by other states must take into consideration the 
requirement for including state educators in the quality control aspect of item development and scoring of 
performance tasks. The extent of educator involvement will be discussed during negotiations and agreed 
upon prior to contract execution. 

To the greatest extent possible, the Department is seeking a solution that provides comparability of results 
with other states. Such comparability could be obtained by a number of acceptable means. The assessment 
could be one that is also administered in other states, or the design could include embedding of a small 
number of norm-referenced items within each criterion-referenced test form. A design that uses embedded 
norm-referenced items would not be expected to yield student-level comparative data, rather in this case, the 
Department would expect reporting at an aggregate level. The final design will be discussed during 
negotiations and agreed upon prior to contract execution. 

7.3.1 Test Content 

1. The FSA will consist solely of test items as described in the previous section. If an existing "shelf product" 
test includes additional components beyond those described above, it will be necessary to remove these 
items. 

2. If the test being proposed is to be derived from an existing product, respondents should: 

a. Describe the general content being measured by the proposed test. 

b. P rovide full-length inspection copies of the test and test administration manual for each grade level. 

c. Provide a test blueprint that governs construction of the test, including the number of items per 
objective and reporting category. 

d. Describe the objectives being measured by the test. 

e. Describe the taxonomic classification system used to classify each test item and the proportional 
assignment of items within the taxonomic classification system. 

f. Describe the range of content difficulty included on each level of the test including the extent to which 
the items at each level assess content that may be difficult for some students. 

g. Describe the method by which text complexity and grade-level appropriateness was determined for 
reading passages 

h. Describe how grade-level appropriateness of item content for each level of the test was determined 

i. Describe the extent to which the items included at each level assess higher order thinking skills. If 
the taxonomic classification system (See e. above) includes a thinking skills dimension, please 
include this information in that description. 

j. Describe the manner in which the test is configured into subparts for both test administration and 
score reporting purposes, including how reporting categories are related to the taxonomic 
classification system. 
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k. Provide an indication of the minutes of testing time required for the separate content sections. 

3. The respondent should describe the special materials or manipulatives required or permitted for correct 
administration of the proposed test (e.g., online rulers or Department-approved calculators). Detailed 
descriptions of or access to each required item should be provided with the technical reply. 

4. Reading passages should expose students to grade-level text of appropriate complexity. Texts should 
reflect a balance of authentic literature (stories, drama, and poetry) and informational text/literary 
nonfiction, which would include passages that address the literacy standards requirement for history/ 
social studies, science, and technical subjects that students are expected to read. Selections should 
include a combination of single texts and paired passages of sufficient complexity for close, analytical 
reading which would allow students to discern and locate evidence from the text to support assertions in 
multiple-choice items or writing responses. Respondents may propose passages from three sources, 
including public domain, permissioned, and commissioned sources. Solely for cost comparison purposes, 
respondents may assume a test design that includes 1/3 of each passage type, but respondents may 
propose solutions that best meet the primary requirements of the ITN. The final test design will determine 
the proportion and genre percentages for passages. 

5. English Language Arts/Literacy items aligning to each passage may include multiple-choice items as well 
as short, constructed responses to text-dependent questions. One extended response item should be 
drawn from a paired or multiple-text passage set. Extended writing prompts should allow students to 
produce opinions/arguments, informative/explanatory responses, and narratives. 

6. The first tests to be administered by the selected contractor under this ITN will be the spring 2015 E LA/L 
and Mathematics tests in Grades 3 - high school. 

a. The selected contractor will be required to provide stand-alone computer-based calculator 
applications for 4-function, scientific, or graphing calculator use on computer-based tests, depending 
on final test design. These computer-based calculator applications will be made available to districts 
and schools for downloading to local computers and for online access and practice throughout the 
year. 

b. For any Florida test produced for students, the selected contractor will provide braille, large-print, 
regular-print, and one-item-per-page versions at each grade level for students with disabilities. The 
contractor must also provide computer-based accommodations for students with various disabilities, 
as described in Section 7.3.11.2. A demonstration of these accommodations may be included in the 
respondent's presentation. 

c. The selected contractor will also print and ship Braille and large print sample test materials upon 
district request for students who will require these paper-based accommodations. These sample test 
materials are considered "generic," meaning once they are created by the contractor, there is no 
expectation that they will be updated during the life of this contract. 

7.3.2 Test Configuration 

This section provides information on the configuration for all test documents and related materials for both 
print and computer test delivery. 

7.3.2.1 Test and Answer Document Design 

Respondents should describe the proposed document design and layout for each grade and 
subject test, for computer-based and paper-based tests, as described for each year in Appendix A. 
The selected contractor designs and produces camera-ready art for the test forms that are used to 
print test books and answer books or prepare computer-based test items in delivery-ready format. 
Contractors are expected to provide the necessary human and other resources necessary to meet 
extremely critical deadlines for delivery of documents and the computer-based test system and 
files. 
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Respondents have been requested to submit bids for a paper-based and computer-based format 
for each assessment. However, even if a computer-based assessment is the accepted solution, 
any tests offered on the computer must also be prepared for use in a paper-based format (Braille, 
large print, regular print, one-item-per-page) as noted in these tables. In this instance, the paper- 
based format will only be offered by special request for up to five percent (5%) of the students and 
can be used at the Department's discretion. 

To the extent that a respondent's design solution allows for some customization, the Department 
will consider replies that include: 

1. Security-controlled booklets containing only FSA Reading passages and no test items for the 
computer-based tests to be available by special request for up to five percent (5%) of the 
students. See Section 7.3.11.1. 

2. Blank work folders for all students taking the FSA Mathematics computer-based tests. The 
work folder consists of four pages for students to do scratch work, the fourth page containing a 
full-page grid. 

3. Blank worksheets for all students taking the FSA ELA/L computer-based tests. The worksheet 
consists of two pages for students to make notes. 

4. A copy of the appropriate reference sheet, if proposed by the respondent and accepted by the 
Department, may also be provided for students taking the assessment for which it is proposed. 

5. Work folders, worksheets, and reference sheets/periodic tables are not numbered or 
inventoried by the contractor; they are destroyed by the school after results arrive. 

For respondents who do not offer such customization, replies should address how test design and 
presentation eliminate the need for these considerations. 

The Department may choose to modify the design of any state-specific test documents/files prior to 
any test administration, and reserves the right to change this configuration beyond the 
specifications but within the scope of the contract, if necessary, through change orders or contract 
amendments. Appendix E provides additional details concerning these materials. 

7.3.3 Develop Items 

For items developed after contract execution, it is the responsibility of the selected contractor to ensure that 
the quantity of items developed and submitted for review annually is sufficient to create operational test forms 
for each administration. Respondents should propose item and passage counts by item type, standard or 
cluster, as appropriate, needed to be developed annually in order to meet these needs. The contractor must 
identify and recruit a sufficient number of highly-qualified writers for passages and items to ensure variety and 
quality of development. Resumes for these writers should be provided and approved by the Department prior 
to contract execution. Note that no active Florida educators will be approved as item writers. 

The contractor must conduct in-depth training of passage/item writers at least annually. Department content 
experts will attend the passage/item writer training to provide input and oversight for the process. 

The selected contractor will be responsible for internal reviews to ensure item clarity, content accuracy, 
adherence to Test Item Specifications, psychometric properties, difficulty, readability requirements, and 
potential bias or insensitivity. It is unacceptable to submit to the Department proposed test items at any time 
that contain blatant errors, such as violation of Test Item Specifications, spelling errors, or grammar mistakes. 
The Department will not review any submissions with these types of obvious errors even if the impact creates 
schedule challenges for the contractor, for which the contractor will be responsible. 

7.3.4 Review Meetings 

For test items developed for use on assessments following the spring 2015 administration, the selected 
contractor will be responsible for organizing and convening meetings for the purpose of reviewing assessment 
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items for content and bias and community sensitivity issues by Department-appointed review committees 
composed of Florida educators and other citizens. A detailed list of meetings to be conducted under this 
contract and the funding mechanism for these meetings are provided in Sections 7.7.7 and 7.7.9 of the ITN. 
For each meeting, the selected contractor will be required to maintain original forms of all committee member 
test security and non-disclosure agreements. These will be made available to the Department on an as- 
needed basis. 

The meetings will be held in a Florida location that is easily accessible (hub cities) to the majority of 
participants and capable of handling the logistical requirements of a large-scale, multi-committee meeting. 

The selected contractor will prepare assessment items and all other materials required to conduct the 
meetings. Separate content review meetings will be conducted for each subject/grade combination. 

If more than one contract is awarded as a result of this ITN, the Department will coordinate with the 
contractors to determine the most efficient method of planning and conducting the review meetings. The 
respondent may propose recommended solutions in the event of the award of multiple contracts. 

7.3.4.1 Bias and Sensitivity Committee Reviews 

The selected contractor is responsible for coordinating all logistics and for preparing materials for the 
bias and community sensitivity review committees. The committees are normally composed of 
representatives of Florida citizens and educators from various backgrounds who review each newly 
developed test item. The primary purpose of each committee is to consider whether the subject 
matter and language used is free of potential bias and acceptable to Florida students, parents, and 
other community members. 

Respondents should propose methods for conducting these reviews. 

7.3.4.2 Item Content Review Committees 

For the subject/grade combination content review meetings, each item should be projected on a 
screen and changes should be made to items onscreen throughout the review process. The selected 
contractor will be responsible for supplying the computers, projection devices, and other hardware 
and software needed to project and modify items onscreen. The contractor will also provide 
calculators, reference materials such as Test Item Specifications, textbooks relevant to the grade and 
subjects being reviewed, dictionaries, and thesauri for each room as is appropriate to the nature of 
the meeting. Print copies of the items being reviewed will be provided to each participant in security- 
controlled notebooks. 

At the conclusion of each meeting, Department and contractor staff will review and evaluate changes 
and ideas recommended by each committee. The Department will have final approval of all changes 
to be made. 

After these meetings, the selected contractor will conduct an editorial and content review of the items 
to ensure that changes have been applied correctly and that all editorial and content issues are 
corrected. This process is referred to as the "Cleans" review process. The Department will review 
and have final approval of all items. 

The selected contractor's development project manager, the development team, and necessary 
assistants will participate in the assessment item content review meetings to manage the meetings 
and to be responsible for organizing, distributing, secure material tracking, and data entry functions 
required for meetings. Arrangements will be required for collecting and keeping secure materials 
locked in secure storage when not in use and for copying materials during the meeting. 

7.3.4.3 Data Review Committees 

Once items have been field tested, the contractor is responsible for convening a meeting to review 
the statistical data on each item from field testing to identify any issues from a psychometric 
perspective. This Data Review Committee may include state content experts, grade-level teachers, 
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accessibility and accommodations experts, and psychometricians, as appropriate. The primary 
purpose is to ensure that all dichotomously-scored items and tasks behave as expected, include all 
correct and appropriate responses, and are fair for all students. This committee may review 
differential item functioning (DIF) statistics to ensure that an item did not unintentionally favor or 
disfavor a particular subgroup of students. Additional considerations include determining whether an 
item can be used as is or needs to be revised and, if so, whether it will be returned to the field test 
pool for further testing. The contractor will be responsible for applying any recommended edits to 
items and will submit the revised items for final Department approval. 

7.3.4.4 Content Advisory Committees 

Of particular importance for the design and development of Florida assessments are the Content 
Advisory committees. The members of the ELA/L and Mathematics Content Advisory committees 
advise the Department about the scope of the assessments for these areas. Their recommendations 
may include: standards to be assessed; the number of benchmarks, passages, and items to be 
addressed per grade level; the item types recommended for each assessed standard; and the 
content of the Test Item Specifications documents. 

7.3.5 Develop Field-Test Item Sets and Forms 

For designs using embedded field testing, it is the responsibility of the selected contractor to group the 
developed items into item sets that will be included in the operational forms of the test for each content area. 

In making the decision as to which items will be field-tested, the selected contractor's field-test proposals must 
consider the areas of greatest need in the item bank, the range of content and rigor included in the items, and 
the number of items available to be field tested. 

7.3.6 Develop Stand-Alone ELA/L Field-Test Item Sets and Forms 

Depending on the measurement model for the proposed ELA/L assessment, the Department will consider a 
stand-alone field test for ELA/L extended constructed-response tasks. Respondents who propose a stand- 
alone field test should prepare field test sampling specifications detailing the sampling methodology. The 
Department prefers that such a field test, if proposed, take place in the December time frame of each year of 
administration. A field test administration is not listed in other sections of this ITN (schedules, test 
administration manuals, etc.); however, if it is determined during negotiations that a stand-alone field test will 
take place, all test administration and scoring schedules and deliverables will be agreed upon prior to contract 
execution. 

7.3.7 Constructing Test Forms 

This section describes the requirements for constructing test forms, including the creation and maintenance of 
test construction specifications. The test construction process must be supported by sophisticated computer 
software, preferably a component of the item banking system that will generate test statistics and related 
curves. The construction of operational forms must be completed in time to meet the requirements of this ITN 
for delivery of test materials to districts. 

7.3.7.1 Test Construction Specifications 

The specific processes, guidelines, requirements, and schedule to be followed in constructing all 
forms of the assessments are to be provided to the Department as detailed, expert-developed, and 
referenced specifications. The test construction specifications must also include information about 
the criteria for item selection, content representativeness, design of the assessment, evaluation 
criteria for reviewing statistical characteristics of items, and desirable assessment characteristics. A 
section of the specifications must address considerations and requirements for those assessments 
that are pre-equated, if applicable, as well as a section for unique requirements of constructing 
computer-based assessments. Careful and extensive development of the creation of linking or 
anchor item sets must be included with details about statistical and content requirements for selecting 
sets of items and criteria for annual refreshing/replacement of portions of the anchor item set. 
Additionally, the specifications must include necessary information regarding item context/position 
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based upon previous use. The selected contractor must also retain assessment construction records 
during the lifetime of the contract. These records must include assessment summaries (item IDs and 
item/assessment statistics) of the various versions of proposed assessments and reasons for 
decisions made regarding each version, and Department approval documentation for the final version 
of each assessment. 

7.3.7.2 Test Construction System 

The contractor will provide a computer-based test construction system that interfaces with the item 
banking system to select and sequence items, generate test curves, and provide statistical summary 
tables and test curves upon demand. The system proposed by the respondent should be fully 
described in the reply including technical specifications, screen shots and access to a demonstration 
version of the system. 

7.3.7.3 Construct the English Language Arts/Literacy, and Algebra 1, Algebra 2, and Geometry 
Retake Forms 

Grade 10 English Language Arts/Literacy and Algebra 1, Algebra 2, and Geometry retake tests will 
be administered under this contract beginning in fall 2015. Respondents should propose pre-equated 
retake forms that can be scored for reporting on accelerated timelines. Solutions for retakes that do 
not include performance tasks will be considered, even if performance tasks are included in the 
operational forms. Overall test characteristics must mirror those of the operational assessments; 
however, the Department will discuss the design of the retakes during negotiation and a decision will 
be finalized prior to contract execution. 

7.3.7.4 Prepare Camera-Ready or Screen-Ready Test Forms 

For each grade or subject test, the selected contractor is responsible for preparing camera-ready 
documents or screen-ready files after the test items have been selected, composed in forms per 
grade/subject, and reviewed. Preparing test documents for publication includes developing test book 
configurations, test maps, specifying page layouts, proofing composed PDFs, and completing digital 
proofs in conjunction with the Department and external reviewers, as appropriate to the selected 
contractor's proposed solution. 

7.3.7.5 Interpretive Products 

The selected contractor will develop interpretive products (IP) to interpret or describe test items for 
specific subject areas or to explain test results. The contractor will be expected to propose sample 
items, review publication PDF and digital rounds, provide alternate text for ADA-compliant files, 
create or refine annotations for student responses, and ensure coherence and consistency with the 
operational test. Interpretive products follow the production and review guidelines outlined in Section 
7.3.10. Required products include the following: 

Understanding Florida's Assessment Reports 

This publication describes the various reports of test results distributed to students, schools, and 
districts. An explanation and an image of each type of report are provided. The booklet also includes 
information about the test content areas and a glossary of technical terms. The contractor will deliver 
the electronic ADA-compliant PDF for posting to the DOE website so files can be posted to coincide 
with the first results delivery (RD1). 

Sample Test Materials (STMs) 

STMs will be delivered for all assessments, including assessments that are primarily computer- 
based, and include 10-20 developed sample items in a sample test booklet for students with a 
separate answer key booklet for teachers. Both documents are produced in ADA-compliant PDF files 
for unsecure web posting and in large print and Braille accommodated versions. The printed 
accommodations are delivered to schools each fall so students can practice answering various item 
types. The ADA-compliant version is delivered at the same time so all students have the same 
opportunity for practice. Respondents should propose designs for sample test books and answer 
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keys for the ADA-compliant PDF version and for printed large print and Braille accommodated 
versions. 

English Language Arts/Literacy and Mathematics Scoring Guides 

The contractor will be expected to post electronic publications that provide information to help 
educators understand the scoring of constructed-response items included on student reports. These 
publications provide information about the test design, standards and benchmarks, item types, and 
how items are scored. They also provide scoring rubrics and the annotated papers used to train 
scorers for the constructed-response items. Each sample item from the scoring papers is displayed 
as it appeared on the test and responses for each score point, images of student responses, and 
annotations explaining why the response received a particular score are provided. 

Computer-Based Test (CBT) Practice Tests 

Respondents should specify a plan for delivering computer-based practice tests for all computer- 
based assessments, including accommodated computer-based assessments to be reviewed and 
revised as needed in conjunction with Department and external reviewers. The purpose for these 
practice tests is to familiarize students with the computer-based testing system and to provide 
students with an opportunity to practice answering various item types. 

All practice tests will include items similar to test format and content. The practice session may 
combine subject items or use separate practice sessions for each subject area as the Department will 
determine. Practice sessions will include approximately six (6) to twenty (20) items per subject. 

Review and development of the student tutorials/practice sessions must follow the Department's 
procedures for forms review, outlined in Section 7.3. In addition, the Department reserves the right to 
request updates to the practice sessions to reflect any changes in CBT platform functionality or 
necessary changes to the practice questions throughout the life of the contract. The contractor must 
anticipate that these updates may occur and must be prepared to provide these updates at no 
additional cost. (These would not qualify as "author's alterations" defined in Section 7.7.9.3.) 

Additional information about the computer-based format and practice tests is provided in Table 
7.4.9.6. 

Test Item Specifications 

Respondents should propose a detailed plan for producing and reviewing test item specifications for 
ELA/L and Mathematics grades and subjects in consultation with the Department and Florida 
educators. The specifications govern the writing and review of passages, prompts, stimuli, and test 
items and generally identify the item/task requirements and constraints for measuring each standard. 
The publications may include introductory information and appendices as necessary to fully explain 
how the tests and items are developed. The test item specifications may include clarification of the 
standard being assessed, specific content and item limitations, and item formats by which each 
standard will be assessed. Sample items may be included for each item type. Item writers are the 
primary audience for these publications, but Florida educators also use these materials, so ADA- 
compliant PDF files will be posted to the Department website. These publications will not be 
distributed in print. Respondents may propose approximate document lengths and document design. 
In conjunction with the review of the test item specifications, an external review panel of Florida 
educators will review the test design and philosophy contained in each manuscript and will make 
recommendations for improvements to the item specifications for each assessment. 

7.3.8 Production of Printed Test Materials 

1. The selected contractor is responsible for producing all versions of the materials and products developed for 
this project, including test forms. 

2. The selected contractor is responsible for proofreading documents during all stages of production. The 
contractor should propose well-defined procedures to ensure drafts are checked by computer for spelling and 
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proofed for complete accuracy by qualified proofreaders from the respondent's staff before they are sent to 
the Department for review. Qualified persons within the respondent's organization and independent editors, 
who will be responsible for proofreading, should be named in the reply and their resumes attached. The 
respondent should include a provision for the cost of changes made during the final proof stage of production 
as "author's alterations" to be authorized by the Department. 

3. The selected contractor will design and implement a procedure for assessing the quality of printing. The 
procedure must include on-site quality control plans to ensure 100% correct printing, sealing, and collating test 
documents. In their replies, respondents should explain what arrangements they have reached with printers 
to establish technology or procedures to prevent miscollating test and answer books. Printing subcontractors 
that produce miscollated test and answer books may not be approved for subsequent test administrations. 
Procedures for ensuring printing quality must include plans for proofing all materials before and after printing. 
The selected contractor will be responsible for replacing misprinted or otherwise defective materials at the 
contractor's expense within a time period acceptable to the Department 

4. In addition to the quality of printing, the Department requires stringent test security procedures to be followed 
during all stages of document production and printing. In their replies, respondents should explain the security 
plan to be implemented while test documents are in production and at the printer. See Appendices F and I for 
these restrictions. 

5. If errors occur during production, the selected contractor will be responsible for providing to the Department 
within 30 days of the eventa root-cause analysis, providing details on the cause and steps for improvement. 

7.3.9 Production of Computer-Delivered Products 

1. The selected contractor is responsible for producing all versions of computer-delivered products developed for 
this project, including test forms. These products include all information, text and graphics that will be 
presented to computer-based test users. 

2. The Department recognizes that the development, review, and approval processes for computer presentation 
of information and materials parallels similar work for printed tests and products. 

3. The respondent should specify its suggested plan for computer-based-test production with the objective of 
ensuring that industry-leading quality is attained. The plan for the test content review should include steps 
required to develop computer-based tests and practice tests to be reviewed and revised as needed prior to 
approval in conjunction with Department and external reviewers, as appropriate to the respondent's proposed 
solution. 

4. The functionality of CBT forms (e.g., tools, reference sheets, navigation) must be checked carefully to ensure 
that errors have not been introduced, and that the functionality remains consistent across forms and 
administrations. The Department will provide a checklist that can be used as a starting point for the 
functionality review, but the selected contractor must be prepared to devote significant time to thoroughly 
check each form. 

5. A final review round will include an "end-to-end" check of the product, including checks to ensure user access 
to the product and accurate navigation and functionality. The respondent should include a provision for the 
cost of changes made following approval of a computer-based-test form as "author's alterations" to be 
authorized by the Department (see Section 7.7.9.3). 

6. Qualified persons within the respondent's organization and independent editors, who will be responsible for 
proofreading, should be named in the reply and their resumes attached. 

7. The Department requires stringent test security procedures to be followed during all stages of document 
production and printing. In their replies, respondents should explain the security plan to be implemented while 
test documents are in production. See Appendix F for these restrictions. 
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8. If errors occur during production, the selected contractor will be responsible for providing a root-cause 
analysis to the Department within 30 days following the event, providing details on the cause and steps 
for improvement. 

7.3.10 Production of Paper-Based and Computer-Based Test, Style Guide, and Braille Production Procedures 

1. The selected contractor will be responsible for all aspects of production for the CBT system, secure web- 
based reporting site, and printed materials, including formatting, graphics, and key entry. For each 
publication, the contractor will submit product specifications that identify printing plans, type size and 
style, ink and paper color, paper quality, and layout. P rinting examples that show type size and style will 
be included. The contractor will provide copies of the final electronic files for products. 

2. The selected contractor will also print any additional materials needed to implement the project, such as 
transmittal memoranda, labels for packing and packing lists. 

3. The selected contractor will be responsible for all aspects of production for publishing printed products, 
including formatting, graphics, and key entry. For each publication, the contractor will submit printing 
plans that identify type size and style, ink and paper color, paper quality, and layout. Printing examples 
that show type size and style will be included. The contractor will present these plans annually. The 
Department expects attractive, quality printed materials at reasonable cost. Guidelines for the quantities 
of each product to be printed are indicated in Appendix E . 

4. Editorial specifications must include editorial and stylistic considerations for the development of test 
questions and the layout of test pages. The selected contractor is responsible for updating and revising 
the specifications annually as style and layout requirements may change. The contractor will work with 
the Department to identify the components of the specifications that need revision and will revise and print 
copies for all users. 

5. The selected contractor will submit plans that identify screen layouts and navigation elements, font style 
and size requirements, background and foreground color palettes, and requirements for graphics. The 
contractor will identify and propose a branding plan to identify and provide consistency for Florida 
assessment components delivered via computer. The Department expects an attractive, high quality, 
engaging, and functional presentation. The contractor will present this information, as well as the 
proposed scheduling and process for review rounds, in a specifications document that addresses 
computer-based presentations. 

7.3.11 Produce Special Formats and Accommodations for Students with Disabilities 

7.3.11.1 Paper-Based Accommodations 

For all assessments, the selected contractor will provide Braille and large print versions for visually 
impaired students. For all assessments, the selected contractor will also provide one-item-per- 
page/one-sided accommodated versions as unique accommodations. For test documents, the 
selected contractor will create the electronic files in the correct format needed to produce these 
special formats. Although multiple forms of the tests may be constructed for each administration, 
only one form for each test will be converted to Braille, large print, and one-item-per-page versions. 
Large print documents will be printed in a minimum of 18-point type on approved paper and bound 
using agreed upon methods. Reformatting of documents may be necessary to meet these 
specifications. The selected contractor is responsible for having Braille materials proofed by an 
independent party that includes a certified Braille reader. The Department may also employ the 
services of a Braille proofreader. See Appendices E and G for more details and specifications on 
special formats of test documents. 

1. Respondents should include proposed subcontract publishers of Braille and large print materials 
in their response. This publisher will produce the accommodated versions of the test and 
answer books and other documents at the contractor's expense. Test administrator notes and 
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scripts to accompany each administration's Braille test versions will also be developed by the 
contractor. 

2. While most blind students in Florida require contracted Braille, there will be some students who 
require uncontracted Braille. The selected contractor will be responsible for providing either 
format based on district requests. Approximately 50 copies of the Braille versions and up to 400 
copies of the large print version may be required for each grade level and subject area per 
administration. Test administrator notes and scripts to accompany each administration's Braille 
test versions will be provided by the contractor as a supplement to the manual, and will undergo 
the same iterative review process as other printed materials. 

3. If requested by districts, the selected contractor will also provide large print and/or Braille 
versions of sections of the test administration manuals. These publications will be produced so 
that they will be delivered to districts along with the shipment of the regular print versions of 
these products as identified in Appendix E. A maximum of five (5) each per format may be 
produced for each administration. 

4. The selected contractor will maintain and update an internal document that identifies which 
person, by position and name that will perform each task in the Braille materials process. Each 
task must be described in detail and the document must be accessible online with other 
downloadable materials. 

5. On request, the selected contractor will develop and deliver one-item-per-page documents to 
districts. Approximately 25 copies of the one-item-per-page version may be required for each 
grade level per administration. The need for these specialized documents will be determined by 
the Department through its unique accommodations requests process and the Department will 
provide an approved list identifying students needing this accommodation to the contractor. 

6. The one-item-per-page documents will be packed and shipped by the selected contractor for 
identified students. These documents will be printed on 8 Vi'y. 11" paper, three-hole punched, 
and placed in a binder so that individual pages may be removed as necessary. Because these 
are also used for students who require true black and white print, the selected contractor will 
provide for the Department's approval a sample of the materials to be used that demonstrates 
the paper quality and font/style. The document package will include the appropriate scannable 
answer document for student responses and the contractor will include directions for 
administration and management of this test version. The contractor will be responsible for 
transcription of one-item-per-page student responses. 

7. The respondent's reply should describe the processing of large print, Braille, and one-item-per- 
page versions of answer documents, including handling and transcription during the scoring 
process. All Braille, large print, and one-item-per-page materials require separate packaging but 
must be shipped so that they arrive in districts at the same time as the materials for regular 
students. 

8. The contractor will also deliver paper-based regular print or large print Reading Passage 
Booklets for computer-based Reading tests. The booklets, which contain reading passages but 
do not contain test items, will be for use by eligible students who need to utilize handwritten 
strategies in order to respond to test items on the computer. See Section 7.3.2.1. 

7.3.11.2 Computer-Based Accommodations 

Florida's computer-based tests must also be available in paper-based format as an accommodation 
at all grades and subjects. The use of this format will be dictated by a student's IEP or Section 504 
plan. The Department anticipates that the number of students taking a paper-based test (PBT) 
based upon allowable accommodations will not exceed five percent (5%) of the total population for 
each grade and subject tested on computer, and the selected contractor will produce this quantity five 
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percent (5%) of paper-based tests. For these students, paper-based accommodations described in 
Section 7.3.11.1 must be available. 

The reply should address the capability of the computer-based system to provide the following 
special accommodations for students and to handle any additional requirements related to the use of 
these accommodations. 

1. Screen Reader 

The respondent's reply should include a description of a screen reader for computer-based 
tests. The respondent should propose a method for reading text within the computer-based 
system that does not require additional, proprietary software installed on the user's computer. 
The screen reader should be able to read all text and/or text descriptions of graphics or pictures, 
including graphics or pictures used as answer options. The system should also have the 
capability to use modified text if necessary (e.g., reading an algebraic equation with grouping 
symbols), and allow user control of the reading speed and volume. The screen reader should be 
designed for use by both sighted and visually impaired students. The Department recognizes 
that the contractor may have different types of screen readers for these two purposes, but this 
difference should be transparent to the user. The Department must approve the usability and 
voice of the screen reader. In addition, the contractor must ensure that the pronunciation of the 
screen reader is accurate, consistent across forms and subject areas, and would be 
recognizable to students. If the screen reader does not have accurate, recognizable 
pronunciation, it must be modifiable to meet the Department's standards. In addition, the 
contractor must check the pronunciation on every form to ensure it is correct, is tracking 
properly, and is understandable prior to it being provided to the Department for review. The 
Department prefers that the contractor adhere to the Department's screen reader specifications 
to ensure consistent performance of the screen reader. However, the Department will review 
screen readers that meet the intent of this section and the needs of the Department. 

2. Increased Font Size (Large Print and Zoom) 

The respondent's reply should describe available options on the CBT system to enlarge font 
size, graphics, and CBT interface (i.e., buttons, tools, cursor, scrollbar) for students with 
disabilities and describe how these options are set for a student. See Section 7.4.9.6 for large 
print and zoom size requirements. 

3. Font and Background Color (Color Contrast) 

The respondent's reply should describe available options on the CBT system for changes to the 
font color and background color in order to optimize the ability of students with visual 
impairments to read the screen and describe how these options are set for a student. See Table 
7.4.9.6 for the font/background color combinations. 

4. Assistive /Adaptive Devices 

The respondent's reply should describe the ability of the CBT system to interface with 
assistive/adaptive devices. The reply should include a description of specific devices that may 
be used and known devices that are not supported, as well as any special requirements 
necessary for use, and should describe how options for use are set for a student. 

5. English-to-Heritage Language Translation Dictionary 

The respondent's reply should describe the provision on the CBT system of an on-screen 
English-to-heritage language translation dictionary for English Language Learners, if this can be 
made available. The accommodated student should be able to select a word and be presented 
with a translation sensitive to the context of the text. The language translation dictionary should 
be available in Spanish and Haitian Creole. The respondent's reply should describe the high- 
level tasks involved, note the general timelines required, including the major/essential 
deliverables and services required. 
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6. Enqlish-to-Siqn Language Translation Dictionary 

The respondent's reply may describe the provision on the CBT system of an on-screen English- 
to-sign language translation dictionary, if this can be made available. The accommodated 
student should be able to select a word and be presented with a translation sensitive to the 
context of the text. The respondent's reply should describe the high-level tasks involved, note 
the general timelines required, including the major/essential deliverables and services required. 

7.4.0 Test Administration 

The contractor is responsible for all aspects of test administration described in this section. 

7.4.1 Pack and Distribute Materials 

1. The contractor is responsible for all arrangements and costs associated with packing, distributing, and 
retrieving physical materials. There must be 100% accounting for all secure materials (e.g., test books, 
test and answer books) distributed to and returned by districts using barcode labeling systems, as well as 
for any physical media (e.g., CDs) used to transmit electronic files. The contractor must guarantee that 
distribution procedures are accurate and make corrections to the system in the event of errors. 
Respondents' replies should include descriptions of the procedures they will use to complete these tasks. 
Respondents should identify carriers they propose to use and the procedures for delivery and retrieval of 
test materials. The Department must approve all carriers. 

2. The contractor will prepare packaging and distribution (P&D) specifications for each test administration. 
The contractor will use the information in the specifications to program packaging systems and train 
personnel involved in the processes. The packaging specifications will include the contractor's 
procedures for packing and distributing materials to districts and retrieving materials from districts. The 
specifications will include a description of how the materials are packaged, how quantities of materials will 
be calculated, examples of packing and inventory lists for boxes sent to districts and schools, methods 
used for distributing and retrieving materials, a description of the procedures used to inventory materials 
as they are returned, as well as any special instructions or administration-specific information. The 
contractor will also include samples of all memoranda required in the delivery/pickup process. For 
reference, the Department will provide the contractor with the most recent packaging and distribution 
specification document at the initial contract management meeting. Proof of Department approval for 
each specification document will be evidenced through email confirmation of approval. 

3. The number of districts and special schools that serve as districts may change slightly during the life of 
the contract. Appendix C provides information about the number of districts, schools, and students that 
will receive test materials. Respondents should anticipate shipping materials to district assessment 
coordinators at approximately eighty (80) separate sites. Materials will be shipped to a central district 
location and distributed to the schools by the district. 

4. Several school districts have contracts for services with school sites that are not geographically located 
within the school district itself. School district personnel supervise the sites and the district assessment 
coordinator provides training for site-based test administrators related to the administration, including 
security, packing, and shipping. To enable prompt receipt and return of test materials, the contractor may 
be required to ship materials directly to and retrieve materials directly from these remote sites. The 
Department will work with the district assessment coordinators and the contractor to identify and approve 
the remote sites to which the contractor will be permitted to ship and retrieve materials without going 
through the district office. For the purpose of preparing replies, respondents should expect that up to 
twenty-five (25) such locations might be identified. 

5. At the beginning of the contract, the Department will provide the contractor with a data file containing a list 
of the districts and schools (names and identification numbers). The contractor will be responsible for 
maintaining this list, updating it, and providing a copy of the data file to the Department at the end of the 
contract or at other times within the contract period on request. 
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6. The contractor will collect preidentification information from districts for each administration via the online 
management system and assume that preidentification will be used for 100% of the school population. 
These files will be used to load student data into computer-based testing systems, as well as produce 
barcode PrelD labels and rosters for paper-based tests. The contractor will establish a system to ensure 
that the labels/documents delivered to districts contain all requested data, reflect the options selected by 
districts, are accurate, and are printed at a level of quality that permits accurate scanning and precludes 
the possibility of smudging, smearing, and/or flaking. Contractors will provide quality control throughout 
the printing process to ensure the quality of label/document printing. The contractor will provide a 
separate roster of students using the same sort order used to prepare the student labels/documents. This 
roster will be used by districts and schools to verify the accuracy of certain information that must be 
correct before the label/document can be used. 

7. A list of the primary materials to be shipped for each administration, the quantities to be packaged for 
schools, districts, and the Department, and other packaging specifications is provided in Appendix E. 

8. The contractor's specifications for packing, distributing, and receiving materials will include the following 
provisions: 

a. The contractor will shrink-wrap materials in quantities specified by the Department. Some materials 
will not be shrink-wrapped (e.g., Test Administration Manuals). All secure materials will be shrink- 
wrapped. 

b. Schools within a district will be ordered by school number on all lists and for shipping purposes, 
unless the district has requested that materials be sorted in route code order, in which case the 
materials will be sorted by route code, by school number. 

c. The contractor will package and ship materials by district size as determined by the Department, 
beginning with the largest district. The largest district will receive materials at the beginning of the 
delivery window; the smallest district will receive materials last. District rank by size information will 
be updated annually by the Department. 

d. The contractor will label the boxes of test materials with the message 'TO BE OPENED BY 
ASSESSMENT COORDINATOR ONLY." District boxes will be differentiated from school boxes by 
color (e.g., white district boxes and brown school boxes). Boxes containing the district/school 
packing lists, materials return kits, and ancillary materials must appear different (e.g., different color, 
colored stripe) from all other district and school boxes for easy identification. School boxes will be 
labeled with the name/number of the district and name/number of the school in at least a 16 point 
font. Label design must be approved by the Department. Only boxes directed to the district-level 
staff, such as boxes containing district overage, will be labeled with the name of the district 
assessment coordinator. The contractor will label district boxes (that contain overage and other 
materials for the district) on all four sides and the top, and number boxes as "Box 1 of X," where X is 
the total number of boxes sent to that district. The contractor will label school boxes on all four sides 
and the top, and number boxes as "Box 1 of X," where X is the total number of boxes sent to that 
school. If materials for more than one administration or type of material are shipped during the same 
window, the contractor must have a way to distinguish these shipments from each other (e.g., color- 
coded label on boxes). Boxes containing secure test materials must never indicate secure content 
on the outside of the boxes. 

e. For districts receiving more than one pallet in a shipment, a map listing the identity and location of 
boxes on each pallet will be provided electronically and will be provided at the time of delivery as a 
hard copy to the district assessment coordinator to assist with distribution of boxes in the district. 

f. The contractor must have a method to collect route code information from districts and package 
materials accordingly. If district assessment coordinators provide internal district delivery route code 
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information, the contractor will use this information to build pallets for districts to match district 
delivery route codes. 

g. The contractor will pay charges on all materials shipped to and from each district. The contractor will 
make arrangements for and pay for shipment if, due to a delivery error, a district is asked to ship 
materials to another district. The contractor must use an overnight delivery service for such 
shipments. The contractor is not responsible for any costs schools may incur in shipping test 
materials from their schools to the district office, unless the need to ship is the result of a packaging 
error by the contractor. The contractor must also have a method for districts to order additional test 
and ancillary materials, as needed, in the online management system. The contractor must be aware 
that requests for additional materials are often critical and such shipments must be shipped overnight 
for early-morning delivery, on request. 

h. The contractor must receive Department approval before shipping materials to districts. Approval will 
be provided after the Department has received and proofed examples of printed products and 
reviewed sample shipments, as described in the packaging and distribution specifications. Approval 
is sent via an Approval to Ship form. 

i. Secure materials will be packaged sequentially in ascending barcode order (for example, if a school 
receives 200 secure documents and 100 documents fit in each box, Box 1 will contain the documents 
with barcode numbers 1-100, and the document with barcode #1 will be at the top of the documents 
contained in Box 1 and the document with barcode #100 will be at the bottom of the documents 
contained in Box 1. Box 2 will contain the documents with barcode numbers 101-200, and the 
document with barcode #101 will be at the top of the documents contained in Box 2 and the 
document with barcode #200 will be at the bottom of the documents contained in Box 2.). Packing 
lists must list all box contents in order as they are packaged with the first item on the list being the 
first item (on top) in the first box. 

j. The contractor will package materials for each shipment by school in returnable boxes (double- 
walled) suitable for multiple uses. The contractor will provide additional boxes statewide for the 
return of materials that are available as an additional order for districts. 

k. No box will be packed to weigh more than 25 pounds, and boxes will be of the dimensions, or similar 
to, those indicated in Table 7.4.1. If test materials require different box dimensions than those listed 
below, the specifications for boxes will be approved by the Department during the negotiation phase. 

Table 7.4.1 Box Dimensions* 



Table 7.4.1 Box Dimensions* 


BOX SIZE 


MATERIALS 


9.5"x 12"x 13", Double-walled 


Regular Print 


9.5"xl2"x6.5", Double-walled 


Regular Print 


19" x 15" x 4.5" 


Large Print/Braille 


19"xl5"x6" 


Large Print/Braille 


19"xl5"x9" 


Large Print/Braille 



* Box height is dependent upon the quantity of materials in each shipment. 



I. Upon completion of the delivery of each shipment, the contractor will provide proof of delivery to the 
Department indicating, but not limited to, a description of the materials, date/time of delivery, and the 
name of the staff member accepting the delivery. Proof of 100% delivery will be used for invoicing 
purposes. 
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m. The contractor will provide fifteen percent (15%) materials overage to each district for paper-based 
test materials and computer-based ancillary materials (work folders, etc.). The Department and 
districts will decide how materials overage will be distributed to districts and schools for each 
shipment. Districts may select from four (4) distinct algorithms for overage distribution: 

■ All fifteen percent (15%) overage to the district 

■ All fifteen percent (15%) overage to the school 

■ Ten percent (10%) overage to the district and five percent (5%) overage to the school 

■ Five percent (5%) overage to the districtand ten percent (10%) overage to the school 

n. The contractor must anticipate that some districts will require an additional pickup after the specified 
pickup date. This may occur in up to 15 districts for as many as 500 boxes for each spring 
administration, and in up to 10 districts for as many as 200 boxes for each retake and End-of-Course 
administration. 

o. Large-print, Braille, and one-item-per-page materials are to be packaged and labeled separately and 
delivered to districts during the same delivery window as the shipment of other test materials in Test 
Materials Shipment 2 (TM2). 

p. Preidentification labels and rosters will be packaged with test materials (included in TM2). If 
applicable, Wave 2 labels will be shipped separately. 

q. It may be necessary for the contractor to ship test materials to districts/schools and the Department in 
multiple shipments per testing window, not including requests for additional materials. The number of 
shipments may vary depending on the administration. The respondent should refer to the 
approximate shipment dates provided in Appendix B. The contractor must be prepared to deliver 
some materials on alternate schedules per district request (e.g., to avoid conflicts with spring break or 
closures) and must have a method to effectively track these requests and ensure that shipping 
companies meet the revised dates. Shipping companies must never contact districts directly to 
arrange alternate delivery/pickup dates unless agreed upon by the Department. 

r. The contractor will prepay charges on return shipments from districts. Return labels, prepaid postage 
labels, instructions, and other materials needed for return will be provided in a Materials Return Kit 
shipped in TM2. 

s. Districts will return answer documents using overnight delivery, if deemed necessary to meet 
reporting deadlines. Materials will be returned according to a pickup schedule developed by the 
Department and the contractor that allows schools enough time to package materials and also meets 
scoring and calibration deadlines. 

t. The contractor will be responsible for mailing or shipping any miscellaneous materials to the 
Department and districts by overnight delivery service or other means, as requested. The contractor 
is required to secure the services of shippers who will provide inside delivery and unload large 
shipments onto loading docks. All carrier vehicles must display the carrier name, and districts must 
be made aware of what carrier name to expect prior to the pickup or delivery date. Districts may or 
may not have a loading dock or the appropriate equipment to support the delivery and pickup of 
materials (e.g., shrink-wrap; hand-trucks). In this event, the contractor's shipper must be prepared to 
provide such services/materials. District staff is not expected to assist in loading or unloading 
materials, but will be available to provide instructions to the carrier. Carriers must communicate with 
district staff face-to-face when delivering or picking up materials, never leave materials unattended, 
and be prepared to protect materials in the event of inclement weather. 

u. The contractor must develop procedures to monitor the receipt of all materials and develop error logs. 
The date materials are received and any errors made by districts in packaging and completing forms 
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must be documented. When problems arise, the contractor must contact districts and the Department 
to resolve them. Error logs will be identified by school and district, and will document all failures to 
follow the established procedures and, if appropriate, how the errors were resolved. The error logs 
will be delivered to the Department within three weeks after the scheduled receipt of materials from 
all districts. 

v. The contractor will use a barcode numbering system to verify all secure materials returned after 
testing. The contractor will provide a written report to the Department documenting the check-in of all 
secure materials, f the contractor's system for barcode verification is demonstrated to be unreliable 
(e.g., materials not scanned properly), each document will be required to be scanned twice, creating 
independent data files that can be matched for scanning errors. Beyond these measures, the 
contractor will be required to manually check answer documents to ensure 100% accuracy of secure 
document check-in. 

w. The contractor will include in its check-in procedures a method of checking for and retrieving used 
answer documents erroneously packaged with used and unused materials, invalidated materials, or 
test books. These "orphan" answer documents will be processed and scored under extended 
timelines and individual reports will be generated and shipped to the districts. 

x. The contractor will produce statistical reports summarizing responses to the online test administrator, 
school coordinator, technology coordinator, and district coordinator comment forms. A summary, 
analysis, and list of recommended actions derived from the comment form responses, as well as a 
data file containing all responses, will be delivered to the Department no later than two (2) months 
after completion each administration. Contractor staff will be responsible for summarizing the open- 
ended responses received from district and school staff. This summary must be a categorized report, 
not simply a list of open-ended responses. 

7.4.2 Missing Materials Report and Inventory 

The contractor will prepare a missing materials report for all secure materials, based on the scanning that is 
completed during materials check-in. The report must be produced after all materials have been scanned, 
including as many orphan and late-returned documents as possible, and must not reflect materials that are not 
missing (e.g., contractor has verified against the soiled/destroyed documents list). The contractor must 
implement quality control measures to ensure the accuracy of the reports. The contractor will produce a 
preliminary missing materials report for Department review. The Department will review and approve the 
preliminary missing materials reports for distribution to districts. The Department requires ten working days to 
review preliminary reports prior to approval. Reports will be prepared for each affected school listing the title 
of the missing products and the corresponding barcode numbers. School reports will be delivered to district 
assessment coordinators with a district-level summary and memorandum, and a state-level summary report 
will be delivered to the Department. Any missing materials returned by districts will be recorded in the missing 
materials inventory maintained by the contractor. The contractor will deliver a final summary report of missing 
materials to the Department by the date indicated in the Deliverables Schedule. 

7.4.3 Disposition and Storage of Materials 

After testing, the contractor will inventory and store unused test and answer books, manuals, and other 
materials for a period of at least six (6) months. During this time, the contractor will be required to ship 
quantities of these materials, as necessary, to the Department and districts. 

Replies should include a plan for secure destruction of excess test materials that requires Department 
approval. However, the contractor will store 100 copies of each test and answer book for each administration 
throughout the life of the project. In addition, the contractor will store some digital record of the CBT 
assessments as they appeared to the students for that administration. Any materials that may be used in 
subsequent assessments will be stored by the contractor throughout the life of the project. Additionally, six (6) 
months after the contract ends, the subcontractors and print vendors will be required to destroy all electronic 
files and used print copies according to the test security requirements approved by the Department. 
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On verification of the individual barcode numbers of all secure materials returned by the districts and 
acceptance by the Department of accurate results files, the contractor will store used documents containing 
student responses, at contractor expense, for a period of two (2) years. Likewise, all student CBT response 
files must be electronically archived for a two-year period. At the end of the two-year period, the contractor 
will ship or destroy the materials according to instructions from the Department. The contractor must submit a 
letter to the Department requesting permission to destroy specific materials. This process is already well- 
defined by the Department and will be shared at the initial contract management meeting. Test security 
requirements will be maintained throughout the destruction process. It may be necessary to retain a small 
number of used documents for a longer time period due to Department security investigations. At the end of 
the contract, the contractor will be expected to ship these materials to the new contractor. 

Destruction of secure documents and CBT student response files must be requested in writing and authorized 
by the Department. The contractor and all subcontractors must submit certificates of destruction that describe 
in writing the specific items/files destroyed. 

7.4.4 Retrieval of Answer Sheets and Test Books 

In addition to the document retrieval specified in Section 7.4.1, the contractor will periodically be required to 
retrieve up to 1200 used test and answer books per grade, per administration from storage. Likewise, 
individual student response files from the computer-based testing system for up to 1200 students per grade, 
per subject, per administration may need to be retrieved. The contractor must maintain information related to 
warehouse location for each school's used materials to allow for retrieval as may be required for Department 
investigations or audits. In some cases, retrieval of answer documents will require the contractor to conduct a 
manual verification of machine scoring. The contractor will be responsible for costs associated with retrieval, 
review, and possible delivery of these materials to the Department. In addition, if errors are found, the 
contractor may be required to re-score and re-report these documents. 

7.4.5 In novative P rod uct Des ig n 

New, innovative designs are required for all Florida-specific products developed by the contractor. Designs for 
publications will include the organization, format, and page or screen layouts for administration manuals, 
reports of assessment results, information publications, and other materials. Designs will also be produced for 
other products, as appropriate. The contractor will produce all graphics, charts, and illustrations for the 
products for which it is responsible and share designs and graphics with other contractors, as required by the 
Department. The design process will include furnishing conceptual art, design development, and final cover 
designs. The contractor will produce all graphics, charts, and illustrations and will secure the full range of 
copyrights and/or permissions that allow for production in print, secure web, and unsecure web format. 

The contractor must arrange for personnel with professional expertise and experience in the areas of 
visual/graphic arts and document/software/web design to work on the design and development of documents 
and computer/online interfaces. If the contractor does not have qualified, professional personnel devoted 
primarily to visual/graphic arts and document/web design on staff, a subcontractor or subcontractors must be 
identified to provide these design services. Respondents should indicate in their replies the qualified 
personnel/subcontractors who will provide these professional services and include their vitae or other 
documentation of expertise and experience in the reply. 

7.4.6 Printed Products - Administration 

This section provides guidelines, including descriptions of and duration of review rounds, for all printed 
products required for test administration services in this ITN. Contractors must provide high-quality, carefully 
edited materials for each round of review by the Department. 

7.4.6.1 Stages of Development of Printed Products - Administration 

The contractor is responsible for producing the first written draft and all subsequent versions of all 
materials and products developed for this project, and for submitting documents for Department 
review. The review period for the Department is seven (7) work days, but longer review times will be 
required if multiple documents are under review simultaneously. For each review round, the contractor 
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will post an electronic version (PDF) by 9:00 a.m. ET on the first scheduled day of Department review. 
If materials are delivered or posted after 9:00 a.m., the review schedule will be revised accordingly. 
Documents for CBT-related materials will be posted to a secure website for review in their presentation 
format 

The delivery of administration materials is a critical task and workflow must be planned carefully. The 
overlap of reviews could create a burden on both the Department and contractor staff. Excessive 
burdens can be avoided with sufficient staffing, advanced planning, and an early start The contractor 
must propose a detailed materials development schedule for managing the workflow and volume of 
documents to be reviewed in each seven (7) day period in a format approved by the Department. The 
contractor must develop a schedule for the production of test administration manuals that follows the 
review schedule of test books/answer documents or computer-based test forms by one round. The 
schedule must consider what is reasonable given the Department's staffing constraints. All proposed 
schedules must be approved by the Department. The number of documents to be reviewed in any 
seven (7) day period must be negotiated with and approved by the Department. The Department is 
committed to reviewing products submitted by the contractor as efficiently as possible. 

The contractor is responsible for proofreading all documents during the production stages. The 
contractor will propose well-defined procedures to ensure that all drafts are checked by computer for 
spelling and proofed for complete accuracy by qualified proofreaders before they are sent to the 
Department for review. The contractor will submit a written report by the proofreaders indicating 
proposed results and actions. Before moving to digital proof stage, the Department will review and 
approve the proofreaders' report Respondents should indicate in their replies the qualified 
personnel/subcontractors/independent editors who will be responsible for proofreading and include 
their vitae or other documentation of expertise and experience in the reply. 

Test Administration Manuals 

For respondents proposing solutions derived from other products, the Department understands that 
certain test administrations must be standardized, however, there are certain Florida requirements 
that must be met. Replies from such respondents should indicate which aspects of test 
administration are considered customizable, and a final determination will be agreed upon during 
the negotiation phase. For instance, the Department may require the production of test 
administration manuals specifically for Florida that include the standard information, or the 
production of Florida-specific supplementary publications that complement the contractor's more 
generic publications. The following information details the Department's expectations for Florida- 
specific stand-alone products. 

For each year of the contract (Fall - Summer administrations), the following test administration 
manuals will be produced: One (1) ELA/L and Mathematics manual for paper-based tests; One (1) 
ELA/L and Mathematics manual for computer-based tests; One (1) ELA/L Retake manual (beginning 
with the first retake administration in 2015); (1) End-of-Course manual. Administration requirements 
and scripts for all assessed grade levels/subjects will be included in the manuals. Instructions for 
Special Format Tests will include scripts necessary for accommodated testing (e.g., Braille, one-item- 
per-page, and others) and will be provided as separate addenda to the manuals. Computer-based 
manuals will also include scripts for the practice tests necessary to familiarize students with the 
computer-based testing system. All manuals are published in print and on the contractor's website. 
Minimally, manuals will contain instructions and scripts for administering each portion of the 
assessments for the appropriate grade levels for that administration. They also include information 
about security of materials, packing and returning materials, receipt and distribution of materials, and 
the Department's test administration and security policies and procedures, which are subject to change 
with each administration. Revisions will be made as necessary to reflect program changes. 
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The contractor must plan to schedule production of test administration manual covers as separate 
mini-deliverables that will eventually be married to the final products. Each cover requires multiple 
review rounds by graphics designers, editors, and content specialists, and careful planning is required 
to meet production deadlines. 

Test Administration Manuals are to be packaged (no shrink wrap) and delivered to districts for each 
administration as a part of TM1 according to the specifications in Appendix E, Part 3. All manuals 
must also be available in a Section-508 compliant (see Appendix E, Section 2) electronic format on 
both the Department's and contractor's websites within seven (7) days of Departmental approval of 
the web-tagged files. The suggested workflow for the Department and the contractor is provided in 
Appendix G to help the respondent anticipate the level of work required on the review rounds. 

7.4.7 Test Administration Ancillary Materials 

For the purpose of this ITN, test administration "ancillary materials" refers to printed or online products other 
than manuals produced by the contractor. Ancillary materials are developed with review rounds and email 
approval. Some ancillary materials will be printed and shipped with test materials (e.g., document count 
forms, return labels, paper bands). Some will be included as perforated pages in the manuals (e.g., security 
agreements, chain of custody form, blank administration record/security checklist, security log). Some will be 
developed and provided in an online formatonly (e.g., commentforms). 

This section contains details regarding the ancillary materials required for administration. These materials are 
designed to ensure overall security, efficient and secure handling of materials, and standardized 
administration activities. In addition to the materials listed here, the contractor will provide, based on 
Department request and approval, a number of memoranda to assist in communications with districts 
regarding test administration processes and requirements. 

The contractor will provide the required publication production and software development resources to 
produce the ancillary materials using current staff that have the necessary expertise and serve in such 
capacities on a full-time basis, through temporary staffing, or by subcontracting. Respondents should identify 
in their replies the staff who will be assigned to develop the ancillary materials and/or subcontractors proposed 
to complete these project tasks, as well as their qualifications (education and experience) to do so. 

Most of the products identified in this section will be provided in formats that permit them to be accessed 
electronically via the internet on both the contractor's and Department's websites. Posted products must be 
available for viewing and downloading and must be provided in an ADA-compliant format, that is, a format 
compliant with Section 508 [29 U.S.C. § 794(d) & 36 CFR Part 1194] of the Rehabilitation Act, as amended, 
and s. 282.601, et seq., F.S. The Department will run standard Section 508-compliance checks on all 
documents to be posted on the Department website to ensure their accessibility. The contractor will be 
required to modify any noncompliant elements and provide compliant documents. The contractor is 
responsible for securing all copyright requirements, if applicable, for materials submitted for nonsecure web 
posting. (See Appendix E, Part 2 for additional information about Department guidelines for Section 508 pdf 
publications.) 

The contractor will work with the Department to identify the specific requirements for files to be made 
accessible on the online management system and will provide the required files in a timely manner. 

The contractor is responsible for developing the following materials for distribution to districts for each 
administration. These materials will be produced following the quantities and specifications identified in 
Appendix E. The contractor is also required to provide various other materials required to implement 
administration activities, including forms, labels, and documents. 

7.4.7.1 Training Resources for School and District Technology Coordinators (CBT) 

The contractor must provide training materials to support CBT test administrations. These 
materials may include screen-shots and will be delivered via narrated PowerPoint web 
presentations, WebEx, training module, or other similar mode through the CBT test platform. 
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These training materials will consist of an outline of key issues to be covered during school and 
district technology coordinator training with explanations accompanied by full-color graphics, 
including screen-shots, representing critical elements or live demonstrations of processes in the 
online management system. 

All training materials are to be made available to districts in a Section-508 compliant electronic 
format on both the Department's and contractor's website within seven (7) days of Departmental 
final approval of the web-tagged files. 

7.4.7.2 Instructions and Materials for Braille Tests 

The contractor must develop Braille scripts and Braille notes for the administration of Braille tests 
for each subject and grade. The braille notes are secure materials and must be handled 
accordingly. The Braille notes and Braille scripts are distributed with the secure test materials, and 
they must be approved by the Department prior to production. In addition, some items may require 
manipulatives to allow access for visually impaired students. These manipulatives are provided, 
prepared, and shipped to the districts by the contractor. 

7.4.7.3 Student Preidentification Labels 

The contractor will produce student preidentification labels for each student to be tested on a 
paper-based assessment for whom preidentification data were provided by the districts. (See 
Section 7.5.2 for more information about the preidentification process.) Labels will be provided for 
each student answer document as needed. Local school staff will be responsible for placing labels 
on student answer documents. 

• One label for each answer document not already preprinted 

• Sorted as specified by each district (may be different for each grade/subject and different for 
schools within a district) 

• Sorted with page breaks as specified by the district/Department 

• 3" x 3 %" adhesive labels or approved equivalent 

• A supplementary supply of labels will be produced for spring administrations (Wave 2) based 
on PrelD information gathered in J anuary 

7.4.7.4 Student PrelD Rosters 

The contractor will provide a preidentification list containing all information included on the student 
PrelD labels for each school in a format approved by the Department. The list is produced from 
the same file used to print student PrelD labels and contains the same information. (See Section 
7.5.2 for more information about the preidentification process.) One row of information per student 
will be used to verify the accuracy of student demographic information. Rosters will be provided in 
hard copies to schools and electronically via the contractor's secure website. 8 W x 11" paper is 
preferred, but the Department may approve 8 Vi'y. 14" if this is required for readability. 

7.4.7.5 Document Count Form (DCF) 

The contractor will provide Document Count Forms (DCF). The DCF is a pregridded, scannable, 8 
Vi" X 11" document used to inventory answer documents returned by each school. The form 
specifies the subject and grade level test placed underneath it and ensures that the correct number 
of documents is scanned. Forms must be preidentified with district/school names and numbers, 
and blank forms will also be sent as this information may also be hand-gridded. The district/school 
number should match the numbers on the answer documents underneath them, and the number of 
documents gridded on the DCF should match the number of documents beneath it. If 
discrepancies are discovered during scanning, the contractor will work to resolve them, as 
described in the Scoring and Reporting Specifications. 
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If the respondent employs a different acceptable method/form for the process described above, the 
Department and the contractor will agree on the final process during the negotiation phase. 

7.4.7.6 Administration Record/Security Checklist 

Security barcode numbers of all secure materials listed in ascending order by product, will be 
provided to a school/district with space to indicate assignment and return of materials. The 
Department expects that the checklist will also include box numbers and section breaks between 
packages and will be provided electronically in an editable format to districts. 

If the respondent employs a different acceptable method/form for the process described above, the 
Department and the contractor will agree on the final process during the negotiation phase. 

7.4.7.7 Comment Forms 

The contractor will work with the Department to create the following ADA-compliant online forms 
that will be posted to the contractor's website for each administration: 

• Test Administrator Comment Form (One for paper-based and one for computer-based) 

• School Coordinator Comment Form (One for paper-based and one for computer-based) 

• District Coordinator Comment Form (One for paper-based and one for computer-based) 

• Technology Coordinator Form (CBT only) 

• CBT Student Comment Form 

The final output of the comment form responses is described in #26 under Section 7.4.1. 

7.4.7.8 Return Shipping Labels 

The contractor will provide preprinted, adhesive, color-coded (as required by the Department) 
labels for district use in returning materials. 

7.4.7.9 Materials Return Kit- District 

The contractor will provide envelopes containing return shipping labels, bills of lading, return 
instructions memorandum, and Braille notes and scripts, if applicable. 

7.4.7.10 Materials Return Kit - School 

The contractor will provide envelopes containing document count forms, paper bands, and color- 
coded return shipping labels. 

7.4.7.11 Miscellaneous Return Materials 

The contractor will provide envelopes and boxes necessary for packaging large-print, Braille, and 
other special program and special format materials for district use in returning materials. 

7.4.7.12 Paper Bands 

Printed paper bands, preferably 4" x 24" adhesive, will be provided by the contractor for bundling 
scorable materials and provided to schools and districts. The adhesive must not cover the entire 
width of the band to avoid adhering to the documents contained in the band. The bands must be 
printed on one side to identify the type of answer document in bundle. Bands must be 8pt. 
Carolina paper or approved equivalent. In addition, paper band quality must be suitable for filling in 
information lines on band with multiple pen/pencil types (e.g., markers, ball point pens, pencils) 
without smearing/smudging. 

If the respondent employs a different acceptable method for the process described above, the 
Department and the selected contractor will agree on the final process during the negotiation 
phase. 
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7.4.7.13 Rulers / Manipulatives / Calculators 

If any test items require the use of a ruler or manipulative, the contractor will produce/provide these 
materials for testing in a reliable, cost-efficient manner. Respondents should describe proposed 
functionality and features of calculators required for Mathematics tests and the Department will 
provide an approved list of calculators for student use. (Contractors will not be required to provide 
hand-held calculators to districts.) Any testing accessories required for computer-based tests must 
be built into the platform and must also be available for paper-based tests. If materials not listed in 
this ITN are needed for testing, they should be clearly identified in the respondent's reply and the 
Department will specify its requirements during the negotiation phase. 

7.4.7.14 Miscellaneous Memos, Forms, Labels, and Other Products 

The contractor will provide the following products: 

• Miscellaneous memoranda to the district coordinators may be necessary for each 
administration. Letters and memoranda will be of a length to be determined by the 
Department, up to five (5) pages each, and length of documents may change at each 
production round. 

• Cover memoranda for all shipments of materials indicating what is being shipped and how it is 
packed. 

• Packing lists must be provided with all shipments (district coordinator receives a hard copy of 
the district and all school packing lists and receives all lists electronically as soon as they are 
available). 

• Pallet lists (maps) must be provided to any district that has materials on pallets. 

• Other miscellaneous memoranda, labels, forms, etc., will be required to implement the 
program. 

7.4.8 Online Management System 

The contractor's online management system described in this section will be the primary resource for all 
student data management (paper-based and computer-based), as well as test administration and reporting 
tasks. The expectations of this ITN are written with the full understanding that capabilities for an online 
management system are expanding quickly. The Department is interested in reviewing innovative procedures 
and techniques for managing student assessment online, which take advantage of the maturation of 
technology in educational assessment through the life of the contract arising from this ITN. Therefore, the 
respondent may submit a reply that varies from the expectations herein as long as the solution meets the 
needs of the Department and the intent ofthe ITN. Respondents should agree to work with the Department to 
amend the specifications, expectations, and requirements to ensure that the Florida testing program keeps 
pace with the expanding capabilities of test delivery and management via computer. Any such variations will 
be resolved and agreed to during the negotiation phase. 

The online management system should be a secure site that requires usernames and passwords and 
accounts should be customizable based on the access a user will have (state-level, district-level, school-level, 
etc.). Depending on the access level, a user should be able to view other organizations in the system (i.e., a 
state-level user may select and view any district or school in the state) and have the ability to make changes 
or alter data in those organizations. 

The Department will provide the contractor with a list of the current names, addresses, email addresses, and 
phone and fax numbers of the district assessment coordinators. The contractor will populate the password- 
protected online system with this information to create initial district-level user accounts. New accounts should 
be able to be created individually or via file upload to create multiple new accounts or make updates to 
existing accounts. 
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The system should allow districts to upload enrollment counts, preidentification files, and to place additional 
orders for materials. Using the Department's Master School Identification (MS ID) file, the contractor will 
populate the system with all active districts and schools. The system must have the ability to add additional 
schools that are not on the MSID file (e.g., private schools), and these schools may require special school 
types as determined by the department. 

Components of the contractor's online systems (e.g., websites, databases) must be submitted to the 
Department for review of content, layout, aesthetic quality, and functionality. The contractor will make any 
requested changes to such systems. The online systems may not be used for contractor advertisements or 
other promotional purposes. The "branding" of these systems should be primarily for Florida's assessment 
system, but may include contractor logos and other information as approved by the Department. 

The system should also accommodate CBT delivery. All systems that identify, prepare for, and monitor 
student test participation must have the capacity to distinguish between paper-based and computer-based 
tests, including all applications of preidentification and enrollment files. These systems must also be able to 
accurately track student participation by delivery mode (paper-based, computer-based, or both) and reflect the 
test delivery mode(s) in each student file. 

The system should be designed to provide user access to various non-secure and secure links including: 

• general system introduction and descriptive information; 

• updates to current operational status and special situations; 

• self-running software to test the suitability of computers for test delivery; 

• self-running software to download a "secure browser" to ensure test and system security; 

• all user guides, manuals, training, and ancillary materials in downloadable format; 

• procedures to add, modify, or remove students or student information in preparation for testing or 
reporting; 

• record test group codes and accommodations for computer-based tests; 

• ability to invalidate computer-based tests; 

• procedures for school and district staff to update contact information; 

• access to an online help-desk chat system; 

• system status website to display operability status of testing components; 

• access to instances of all computer-delivered tests serving Florida K-12 education programs; 

• access to order additional test materials and track shipments; 

• access to packing lists, pallet maps, security checklists (pre-populated with security numbers of secure 
materials), and material summaries; 

• results files and reports; 

• web-based student practice tests; 

• self-paced training resources for test setup and administration tasks; 

• Training center to enable practice for test setup tasks and to run an infrastructure test to test capacity at 
testing sites; and an 

• infrastructure test. 

The system must collect and maintain records for students as they move through Florida's public school 
systems. An individual student's test results must be maintained in electronic archive for the life of the 
contract and must be returned to the Department at contract termination. 
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7.4.8.1 Test Staff Contact Information 

It is necessary for the contractor to maintain accurate lists of district and school information and 
staff associated with Florida public education testing programs as specified by the Department. 
This information must be easy to update by school staff as information changes. This information 
must include each user's security authorization level and provide an accessible log, in viewable 
and printable format, of each user's participation in the system, including type of participation, date 
and time of participation, etc. (see also Section 7.4.1). 

7.4.8.2 Test Staff Passwords 

The system must securely deliver and maintain system access passwords to users as necessary. 
The passwords must access a hierarchical permissions structure available for school, district, and 
state users. The contractor will work with the Department to determine expiration timelines for 
passwords and user accounts. 

7.4.8.3 Test Session Requirements 

The system must have available during all steps in initiating and delivering tests: 

• a school look-up sub-system; 

• a searchable, sortable student look-up sub-system; 

• rosters of students in each test session. 

During the test session, automated backup, recovery, and restart procedures for the system must 
be in place. The system will recover data from any unforeseen test interruption and return the test- 
taker to the point of interruption. The system must also show student status and progress through 
the items in the test. The Department should be able to view uploaded student responses real- 
time while students are completing the assessments in the event of an unforeseen test interruption. 
Contractor and department users must have the ability to restart a submitted test. The tests should 
only be available during specified administration windows. The features and interface of the 
system may be customizable, and the contractor will consider requests for customization from the 
Department. 

7.4.8.4 Security Requirements 

Security of tests and individual test results must be maintained. Access to student test results 
must be restricted to authorized Department staff and school district personnel based on a secure, 
encrypted, password protected system. 

The system must assure proper identification of each student and the accurate matching of the 
student to his/her test results. Preferably, the system must ensure that a student can take each 
computer-based test just once per login unless authorized by the department. If an error is found 
(e.g., student tests with the wrong login information) the system must have the ability to link the test 
to the correct student profile. 

The ability to make changes to data or processes in any part of the system must be based on a 
password-protected, hierarchical permissions structure. 

Respondents should respond to each of the following items: 

• Describe the security controls over all system aspects. 

• Describe the "levels" of security provided in the system. 

• Describe how the ability to limit access to specific system functions or modules is provided. 

• Describe the log-in security method and assurances to maintain individual confidentiality of 
test results. 

• Describe the authentication process to ensure that an individual is the person logging into the 
system (e.g., the student taking the testis who s/he says s/he is). 
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The contractor and Department staff will create a report that includes a checklist of all operable 
aspects of the system to be used each invoice period. The contractor will prepare the report listing 
any malfunctions or unavailability of the system for Department signoff each period. 

7.4.9 Computer-Based Test Administration Overview 

The expectations in this ITN are written with the full understanding that computer-based testing (CBT) 
capabilities are expanding quickly. The Department is interested in reviewing innovative procedures and 
techniques for item development, test delivery, scoring (see Section 7.5.4), and reporting (see Section 7.5.5) 
which take advantage of the maturation of technology in educational assessment through the life of the 
contract arising from this ITN. Therefore, the respondent may submit a reply that varies from the expectations 
herein as long as the solution meets the needs of the Department and the intent of the ITN. Respondents 
should agree to work with the Department to amend the specifications, expectations, and requirements to 
ensure that the Florida testing program keeps pace with the expanding capabilities of test delivery and 
management via computer. Any such variations will be resolved and agreed to during the negotiation phase. 

The computer-based test system should be capable of being customized to meet the needs of Florida 
students, school and district personnel, and the Department. The reply should indicate procedures used to 
ensure that the computer-based tests accurately assess students possessing a diverse range of physical, 
sensory, and cognitive abilities; the system should ensure adherence to universal design concepts. The 
system should be easy to use for all participants, intuitive, and deliver a consistent "look and feel" for all 
computer-delivered test programs provided by the Department. This includes item layouts, navigation tools, 
and administrative controls such as managing participant demographic and performance information, as well 
as CBT setup activities. The contractor will modify any proposed system to achieve these objectives. 

The contractor and Department staff will create a report that includes a checklist of all operable aspects of the 
system to be used each invoice period. The contractor will prepare the report listing any malfunctions or 
unavailability of the system for Department signoff each period. 

If the Department determines that the computer-based system is not performing adequately, exhibits 
malfunctions that may significantly affect the student testing experience, oris unable to meet the requirements 
listed in this ITN, the contractor will propose a solution to the Department, including providing paper-based 
tests at no additional cost, that allows for students to participate in the assessments and for the Department to 
gather the necessary data. 

7.4.9.1 Hardware and Software Requirements 

The system should: 

• Support both wired and wireless connections to the system and all eligible devices that meet 
the specifications in the table in Section 7.4.9.2. The respondent should also commit to 
continued support for future updates similar to those made available on the latest version of 
Partnership for Assessment of Readiness for College and Careers' (PARCC) Technology 
Guidelines that can be found at ( http://www.parcconline.org/technoloqv ). Other assistive 
technologies may be needed for students requiring accommodations. 

• Support eligible devices with capabilities to "lock down" the device to temporarily disable 
features, functionalities, and applications that could present a security risk during test 
administration. Features that will need to be controlled during test administration include, but 
are not limited to, unrestricted Internet access, cameras (still and video), screen capture (live 
and recorded), email, instant messaging, Bluetooth connections, application switching, and 
printing. The contractor will develop, deliver, and continuously improve capabilities to "lock 
down" the device for the applicable operating systems and browsers. 
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• Support the current version within six months of release and two prior versions of the web 
browsers listed below (as of J une 2013). 

Web Browser Support List: 
Apple Safari 
Google Chrome 
Microsoft Internet Explorer 
Mozilla Firefox 

7.4.9.2 Eligible Device Specifications 

At the time of the development of this ITN, the device specifications that are recommended for 
Florida's public schools are provided below. The respondent's platform for computer-based testing 
should be able to perform, at a minimum, on devices that meet the specifications in Tables 
7.4.9.2.A and 7.4.9.2. B. 



Table 7.4.9.2.A. Device Specifications for Desktop, Laptop, Netbook, and Thin Client/VDI Computers 



Table 7.4.9.2.A Device Specifications for Desktop, Laptop, Netbook, and Thin ClientWDI Computers 


Operating System 




Windows 


Windows 7 or newer 


Mac OS 


Mac OS X 10.7 or newer 


Linux 


Linux: Ubuntu 11.10, Fedora 16 or newer 


Chrome OS 


Chrome OS 19 or newer 


Memory 


1GB RAM or greater 


Connectivity 


Computers must be able to connect to the Internetvia wired orwireless networks. 


Screen Size 


9.5 inch screen size or larger 


Screen Resolution 


1024 x 768 resolution 2 or higher 


Input Device Requirements 


Keyboard 

Mouse orTouchpad orTouchscreen 




The input device must allow students to select/deselect, drag, and highlight text, objects, 
and areas. The input device must allow students to enter letters, numbers, and symbols 
and shift, tab, return, delete, and backspace. To meet security guidelines, each 
Bluetooth/wireless keyboard must be configured to pair with only a single computer during 
assessment administration. 




Other assistive technologies may be needed for students requiring accommodations. 


Headphone/Earphone and 
Microphone Requirements 
(Some student 

accommodations may require 
headphones/ earphones (e.g., 
text to speech). 


Headphones/Earphones 
Microphone 


Additional Guidance 


1 Each computer operating in a thin client environment must meet or exceed minimum 
hardware specifications, as well as bandwidth and security requirements. 

Computers must accommodate the 1024 x 768 screen resolution minimum without 
panning. Some netbook computers may have screen resolutions slightly less than the 
1024 x 768 minimum, yet may meet all other minimum requirements. Depending on 
netbook model specifics, school technology administrators may be able to reset screen 
resolution to meet guidelines. 
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Table 7.4.9.2.B. Device Specifications for Tablets 



Table 7.4.9.2.B Device Specifications for Tablets 


Onpratina Svstpm 


S aerifications 
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Android 


Android 4.0 or newer 
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(with 512 MB RAM or greater) 


Windows 


Windows 8 or newer 
(with 1GB RAM or greater) 


Memory 


By operating system 


Connectivity 


Tablets must be able to connect to the Internet via wired or wireless networks. 


Screen Size 


9.5 inch screen size or larger 


Screen Resolution 


1024 x 768 resolution 2 or higher 


Input Device 
Requirements 


Keyboard 

Touchscreen or Mouse 




Due to the onscreen space occupied by a tablet's virtual keyboard, assessments 
will require external keyboards for test takers using tablets so as not to limit or 
obscure the view of test item content and related functionalities when text input is 
required. 




External keyboards must allow students to enter letters, numbers, and symbols and 
shift, tab, return, delete, and backspace. Tablet touchscreen interfaces can be used 
for student interactions with the assessments other than text input, including to 
select/deselect, drag, and highlight text, objects, and areas. To meet security 
guidelines, each Bluetooth/wireless keyboard must be configured to pair with only a 
single computer during assessment administration. 




Other assistive technologies may be needed for students requiring 
accommodations. 


Headphone/Earphone 
and Microphone 
Requirements (Some 
student accommodations 
may require headphones/ 
earphones (e.g., text to 


Headphones/Earphones 
Microphone 
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7.4.9.3 System Performance Requirements 

The system must demonstrate and report that it can meet the following performance requirements 
in a controlled Internet access environment, requiring a bandwidth connection of no more than five 
(5) kilobits per second (kbps) per simultaneous test taker using a caching solution or fifty (50) kbps 
per simultaneous test taker without caching: 

• deliver less than one second mean screen refresh rate for 90% of all users; and, 

• ensure that no users have a refresh rate of greater than five (5) seconds. 

The respondent should describe the ability to support a substantial number of simultaneous online 
users of the system (up to fifty percent (50%) of the current Membership in Florida Public Schools 
as determined through the most recent Survey 2 data under the Education Information & 
Accountability Services (EIAS) found at http://www.fldoe.org/eias/eiaspubs/pubstudent.asp ). The 
respondent should consider any services provided to other states when presenting this 
information. The contractor must continually monitor and report to the Department the number of 
simultaneous online users of the system during student testing periods. 

7.4.9.4 Connectivity Requirements 

The respondent should consider existing and planned communications infrastructures when 
proposing its system. Preferably, school, district, and Department technology architecture and 
computing hardware should not have to be replaced. 

7.4.9.5 Test Site Set-Up/Certification Requirements 

The contractor will develop specifications for, and deliver a system to operate/deliver online tests 
and that serves as: 

• a secure browser, which both [a] locks access to all other computer and web programs, 
applications, and files, and [b] masks and directly accesses the testing host web site; and, 

• a system check, monitoring required computer software and connectivity readiness for testing 
and which can be run from individual machines. 

The contractor will be responsible for assisting schools in certifying that computers are properly 
prepared and can make the necessary Internet connections to successfully deliver tests. The 
contractor will develop and make available an infrastructure test, consisting of mock items, that 
replicates the size of the largest CBT and is run on the actual testing platform (i.e., users log in to 
the testing platform as if they were logging into a live test). The contractor will assist the 
Department in developing a guide to walk users through running the trial on machines that will be 
used for testing to help them identify technical issues and make adjustments prior to live testing. 

The respondent should describe the proposed test site set-up, certification, and system check 
measures in the reply. 
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7.4.9.6 Computer-Based Test Presentation Requirements 
Table 7.4.9.6 CBT Tools and Features 



The following table lists required and preferred tools and features in the CBT platform. 
Accommodated features are required. More information about the test accommodations that must 
be provided for students are provided in Appendix H. 



Table 7.4.9.6 CBT Tools and Features 


Tool 


Description 


Required Features 


Calculator 


Computer-based calculators will be available to students for the mathematics tests. 
Numbers and functions on the on-screen calculator will be activated by mouse-click on 
the displayed button, or by keystroke of the appropriate keyboard key. User-acceptance 
testing will be required to assure that calculations are performed in the same sequence 
and with the same rounding rules applied as the hand-held calculators. Respondents 
should describe proposed functionality and features of calculators to be included in 
computer-based test forms. 


Reference Sheet 


The system must be capable of displaying a pop-up reference sheet for mathematics 
tests. This will be capable of being enlarged and the Department prefers that, when 
open, the reference sheet not obscure portions of the test item. 


Ruler 


If it is determined that some mathematics test items require the use of a ruler, a pop-up 
ruler will be available to students. The ruler will be easily moved and rotated on-screen 
by the student. 




Straight Edge 


A pop-up straight edge, without measurement increments. The straight edge will be 
easily moved and rotated on-screen by the student. 


Option Eliminator 


A feature for eliminating options. However, the feature cannot eliminate the option by 
placing any mark on or in close proximity to the bubble that would be used for marking a 
response. It must allow the student to easily read the option after it has been 
eliminated. Students must be able to undo the elimination. 


Bookmark 


Mark test items for review and navigate easily to return to previous items, including 
those bookmarked for later review. Bookmarked items should be visible on an item 
summary page at the end of the assessment. 


Highlighter 


Highlight text on all tests and to erase the highlighting. If a student highlights in a 
reading passage, the highlighting should remain through all items related to that 
passage until the student erases it. 


Notes 


Allow students to create electronic notes for reading passages if desired. These notes 
must not hinder reading of the passage. Notes should travel for each passage and item 
associated with that passage, then refresh with each new passage. 


Footnote Pop-up 


Reading items require a pop-up footnote. That is, if the student clicks on a footnote 
number occurring in the passage, the text for the footnote appears. Footnotes must 
also appear at the end of the passage. 
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Table 7.4.9.6 CBT Tools and Features 


Tool 


Description 


Student 

Tutorials/Practice 
Sessions 


Each student that takes the computer-based test must have the opportunity to learn how 
to use the tools, select/enter responses, and navigate the system in a short practice 
session with instructions that may be either teacher-directed or independently 
conducted, as described in Section 7.3.7.5. The contractor will produce and make 
available a web-based practice test for each assessment. The contractor may also 
provide access to a selectable practice test at the beginning of any test session 
within the platform. 


Accommodated Features (Required) 


Variable Font 


The system must be able to display items and text using variable font sizes. For 
example, footnotes for words in passages must appear smaller at the end of the 
passage than the text used in the passage. 


Administrator- 
Selectable Zoom 


The system must be able to Zoom in increments from approximately 12 point to 
approximately 72 point available for the student to zoom or for the administrator to 
preset. G raphics must be able to be enlarged also. Any buttons, tools, the 
pointer/cursor, and the scrollbar must also enlarge based on the zoom level selected. 


Administrator- 
Selectable 
Large-Print Font 


The system must allow for selection of a large print font of at least 14 point font and up 
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to 72 point font. The student must then be able to zoom as indicated above. 


Administrator- 
Selectable Variable 
Font and Background 
Colors 


The system must allow for selection from a pallet of colors for font and background 
colors. The Department prefers fonts of black, dark blue, light blue, pink, and yellow 
with backgrounds of white, black, brown, or dark blue. Other colors may be 
recommended. Any buttons, tools, the pointer/cursor, and the scrollbar must also 
change based on the colors/fonts selected. 


Administrator- 
Selectable Screen 
Reader 


Provide an integrated (preferably password-enabled) selectable screen reader capable 
of reading via headset to the student, all text, mathematical symbols, and text within 
graphics. The contractor mustensure thatthe screen reader has one consistentvoice 
and is not operating-system specific. The contractor must run quality control checks on 
all products, test forms, graphics, and items to ensure the screen reader pronounces 
words correctly. 


Administrator- 
Selectable Assistive 


Support assistive/adaptive devices commonly available to support computer input and 
interaction to persons with disabilities. 



7.4.9.7 Computer-Based Test Screen Layout Requirements 

The contractor is required to recommend a set of screen layouts for the presentation of computer- 
delivered test items. (See also Sections 7.3.11.2 and 7.4.8) These layouts must include: 

• prominently displayed student identifying information [name, student ID, etc., as specified by 
the Department]; 

• icons to access the item tools/references noted above; and 

• icons presented with consistent location and look and feel across all screen layout options. 
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Additionally, the system must code and capture in the item bank the basic screen layout used 
to deliver the item on computer to students. 

7.4.9.8 Training Delivery Requirements 

The contractor will develop specifications for and produce a training plan to provide initial training 
and supply materials to support on-going training for audiences including, but not limited to: 

• the Department K-12 Student Assessment program team; 

• other Department staff; 

• district assessment coordinators; 

• school assessment coordinators; 

• school and district hardware/network technical staff; and, 

• school and district test administrators. 

The Department prefers the contractor to deliver in-person training to approximately 500 key 
Department and district personnel in as many as ten (10) sessions of not more than one-half ( l h) 
day each, held throughout Florida prior to the first implementation of the new computer-delivery 
platform, which will occur in the spring of 2015. Additional training should be provided via WebEx 
or similar web-delivered, interactive sessions scheduled at least six (6) weeks in advance of the 
opening of each online assessment window. These training requirements are in addition to the 
training materials described in Section 7.4.7.1. 

7.4.9.9 Technology-Supported Item Types 

The Department is interested in leveraging empirically-validated item formats which take 
advantage of the capabilities of computer technology. Examples of these item types may include: 

• Drag-and-Drop - in which students select from multiple possible responses to select one, via 
mouse-click or key-stroke combination, and then "drag" it to insert (drop) it in an indicated 
blank to correctly complete a mathematics formula, text-editing task, etc. This process may be 
repeated for the same test item so that multiple answers are placed into multiple blanks. 

• Rotate and other image transformation items. 

• Other formats as the contractor may recommend or the Department may request. 

• The respondent may include the capacity to support these types of items in the reply. 

7.4.10 Customer Service and Technical Support 

The contractor will provide customer service via toll-free phone and email between 7:00 a.m. and 8:30 p.m., 
Eastern Time. The service will be staffed appropriately, as needed, with personnel designated to respond to 
assessment program calls and perform other Florida assessment program tasks specific to the assessments 
detailed in this ITN. Customer service staff must document all communications in a log and furnish it daily to 
the Department during each administration or upon request as needed. The contractor will develop a call topic 
matrix, with the Department's guidance, to ensure that calls are responded to in a consistent and appropriate 
manner. 

Beginning not less than four weeks before and extending one (1) full week after each test administration, the 
contractor must use additional customer service personnel to handle computer-based test administration calls, 
and personnel who are able to provide advanced technical support must be immediately available for any 
issues that cannot be resolved by call center staff. The Department is interested in systems which provide 
help desk functionality via telephone and Internet "chat" systems. 

The system should employ a Department auditable job-ticket method of tracking issue types, status, and 
individual client trouble calls/emails including current aggregate status information for resolution. 
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7.5.0 Scanning, Scoring, and Reporting 

This section of the ITN provides a description of the scanning, scoring, and reporting requirements for this 
project. Section 7.5.2 provides details for preparing school and student information, and Sections 7.5.3, 7.5.4, 
and 7.5.5 provide the requirements for the scanning, scoring, and reporting, respectively. The steps 
necessary to verify the accuracy of these procedures are described in Section 7.5.6. 

The contractor must assign experienced staff to complete the scanning, scoring, and reporting tasks and to 
coordinate with the test development, the test administration, and the psychometric staff during various 
phases of the scanning, scoring, and reporting. As described in Section 7.7.2, it is anticipated that one of the 
four project managers will focus on scoring and reporting and one will focus on computer-based testing. The 
one-page vitae for each of the professional personnel must indicate their scanning, scoring, and reporting 
experience within the organization to ensure the critical and extra-critical dates may be met. 

The contractor's scoring and reporting staff must coordinate with the development team to ensure the file 
layouts accurately reflect the final tests; with the test administration team to obtain information about testing 
procedures, schools, and students; and with the psychometric staff to ensure timely file verification and 
transfer during the scoring and reporting process. 

The contractor must verify the accuracy of all files and reports prior to submission to the Department for 
review. The contractor will develop staffing plans to complete its review of files and reports and then provide 
the Department time to complete its review. 

Because of the secure nature of the personally identifiable student information that is used for this assessment 
program, stringent security measures must be in place for the Department and the selected contractor. See 
Appendix I for more information on these measures. 

7.5.1 Scoring and Reporting Specifications 

In April of each year, the contractor will submit the first draft of the Scoring and Reporting Specifications 

for all tests within a school year. These specifications provide details to ensure accurate and timely scanning, 
scoring, and reporting for the following school year. The specifications are initially drafted by the contractor 
using the specifications from the prior school year, updating procedures or products as instructed by the 
Department. This initial draft includes anticipated schedules for planning the next school year, details on file 
naming conventions, as well as the scanning, editing, scoring, and reporting procedures. These specifications 
do not contain confidential information or exact replicas of products; they provide the general requirements 
and processing steps using information from the prior year to assist in planning. If the initial draft needs 
extensive editing, a second draft must be submitted by the contractor. 

These specifications are used to communicate between the contractor and the Department throughout the 
school year with details added as necessary during the year to ensure that procedures and products are 
sufficiently described and any changes in procedures are documented. The contractor will anticipate having 
additional documentation for its internal communications. 

At the initial planning meeting, the Department will provide the contractor with the 2013-2014 Scoring and 
Reporting Specifications with a list of updates for the contractor to make for the initial draft of the 2014-15 
specifications. The reply should clearly indicate how the scoring and reporting staff will coordinate with the 
other staff members to enable the contractor to develop the Scoring and Reporting Specifications each April 
for the following school year while at the same time conducting the current year's spring reporting activities. 

During the school year, the contractor will maintain an addendum to the specifications noting updates, issues, 
or changes to the specifications. This specifications addendum will be used to document changes made in the 
specifications during the school year and to update the specifications for the next school year. If necessary, 
the contractor may be requested to provide the Department with updated versions of the specifications 
throughout the year. The contractor will submit the final specifications addendum at the same time as 
the last closeout file of student results for that school year. 
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After the initial draft of the specifications has been reviewed by the Department and any edits or comments 
have been incorporated by the contractor, development of file layouts and other procedures may begin for the 
first administration in that school year. 

See Appendix B for the critical and extra critical dates associated with the reports described in this section. 

7.5.2 Preparing School and Student Information (MSID and PrelD) 

1. To aid in the accuracy of reporting results, before the initial wave of reporting the contractor must perform 
checks to be sure that school names and numbers are accurate and that each student's scores are 
reported to the correct school. Due to the numerous types of special schools (e.g., home schools and 
virtual schools) and numerous students testing in schools that are different than the schools where their 
scores are reported, school numbers must often be updated after scanning. The contractor must provide 
staff necessary to make updates to student information and school information between scanning and 
reporting when conflicting information has been provided. The contractor will assist the Department and 
districts to ensure that each student's scores are reported to the correct school during the initial wave of 
reporting. 

2. The Department maintains a Master School Identification (MSID) website with active school information. 
The Department will determine dates that the contractor will download information from the MSID website 
to be used for each administration. Due to multiple waves of reporting for each administration, as well as 
reporting annual EOC results, it is important to use consistent school information and to report scores 
using the information from the MSID website. The Department will provide the contractor with any 
changes or updates to any information on this site, such as changing a school name for the annual 
reporting. The contractor will not make changes to school information or add/delete schools unless 
directed by the Department. 

3. To reduce the amount of information a student must provide during testing, for each administration the 
contractor will obtain student information from each district prior to testing. This preidentification (PrelD) 
information includes student demographic information, requests for special print materials, and school 
numbers of the school where the student will test and the school where their results will be reported. 

In the past, the Department has obtained PrelD information in a text file from each district during one time 
period each fall and summer, and two time periods each spring. The PrelD information was then loaded 
into the CBT system or used to create adhesive P relD labels for answer documents. 

The reply should include a description of how the contractor would obtain and update the PrelD 
information to provide the most accurate results. Respondents may propose other options to adhesive 
labels, such as pregridding (preslugging) answer documents. The Department wants to obtain the most 
cost efficient methods that will ensure accurate results. 

4. The Department will determine the dates the districts will be permitted to submit PrelD information, the 
number of PrelD windows, how and which information may be updated, the rules for accepting or 
rejecting records or files, and the final method of preidentifying the answer documents. 

5. The contractor will draft the PrelD file layouts and the PrelD instructions for each administration and 
incorporate changes as requested by the Department. The Department will send the final PrelD 
instructions and file layout to the districts. The contractor will also post the layout on the online 
management system as instructed by the Department. 

6. The contractor must complete numerous steps to ensure the accuracy of the P relD information submitted 
by each district, and the contractor must work with district staff when necessary to ensure that the P relD 
files are properly uploaded in a timely manner with just one unique record for each student in the district. 

a. First, the contractor must be sure that only schools that are active in the MSID file or approved by the 
Department (e.g., a participating private school or community college) are used in each district's 
PrelD file. Then, the contractor must ensure that each field in the PrelD file submitted by the district 
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is completed with valid characters as described in the layout and that the values are reasonable. For 
example, if all records submitted for Algebra 1 are coded as first-time takers in a district or all are 
coded as female, the contractor must alert the district and permit updates as necessary. Also, the 
contractor must compare the numbers of students in each district to the number of students tested in 
the previous comparable administration to identify any issues with the number of records in the 
district's file and take corrective action as necessary. The reply should include a description of the 
system itwill use to ensure the accuracy and reasonableness of these files. 

b. Two days prior to the close of each PrelD window, the contractor will contact the districts that have 
not successfully submitted a PrelD file and aid the district to ensure each district is able to submit its 
data. 

c. During the PrelD window, to assist district staff in reviewing the files as read by the contractor, the 
contractor creates PrelD Frequency Reports with n-counts for various demographic groups for each 
district to review in the online management system. The contractor will also generate a PrelD 
Frequency Report with state totals forthe Department to review. 

d. The district may submit additional files to make corrections during the PrelD window. The contractor 
will draft the format of these reports and make updates as provided by the Department. The reply 
should include examples of the types of reports that might be used to aid districts in reviewing the 
PrelD data and methods to make updates and corrections while ensuring the final files include one 
record per student with accurate information. 

e. After the PrelD window closes, the contractor provides the Department with a state PrelD file 
containing all the accepted records that were submitted by the districts. This file will be in the same 
format as the files provided by the districts, one record per student, except with a unique PrelD 
Number, generated by the contractor, for each student. The PrelD Number is used for each test the 
student is assigned within that administration and the PrelD Number is unique across administrations 
for each school year. For example, if a student in grade 8 is taking ELA/L and Algebra 1 in the 
spring, that student will have one PrelD number for both spring assessments, and that number will 
not be used again during that school year. 

f. If a student's school is incorrect in the district's PrelD file in (e.g., student moved), school or district 
staff will grid the student information on the cover of the answer document or add student information 
using the online management system prior to testing. 

g. If a student will be tested in a school other than the school used for reporting, PrelD labels are sent to 
the school where the student will test or the CBT record is setup where the student will test. This is 
called the Alternate District/School Number. 

h. The contractor will print PrelD Student Rosters and include them with the shipments of PrelD labels 
(or preidentified answer documents) to districts. These rosters list the information on the PrelD labels 
and an explanation of the codes used on the PrelD labels. The contractor will provide hard-copy 
PrelD Student Rosters with the PrelD labels and electronic copies on the contractor's secure website 
as specified in the Scoring and Reporting Specifications. Each spring, via a second PrelD window to 
provide updates closer to testing, the contractor will provide a supplemental shipment of PrelD labels 
and rosters to districts that update their PrelD files. 

i. As permitted by the Department, districts may subsequently edit some of the information using the 
online management system or add new students for the CBT system. Students who were not 
included in the PrelD files that are taking paper tests will not be loaded into the PrelD system, and 
PrelD labels are not sent for new students taking paper tests. 
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7.5.3 Scanning/Editing PBTs and Extracting CBT Records 

Scanning and editing requirements in this section refer to the PBTs. The contractor's CBT system must be 
capable of creating a file with each student's information and the exact responses entered by each student, 
including the fill-in response and performance tasks, character by character before editing, similar to the scan 
files described in this section for the PBTs. The reply should include a description of how the scanning and 
editing procedures in this section will be completed, as well as a description of the types of CBT extract files 
that will be available for the Department prior to the creation of the scored files described in the next section of 
thislTN. 

1. For each administration, the test administration manuals describe in detail the test materials provided to 
districts and schools and the procedures for returning the materials and completing information in the CBT 
system. The contractor will establish procedures to verify that the district and school staff completed 
these procedures correctly. The contractor will work with the Department and districts if any anomalies 
are found and make corrections as necessary. This includes mistakes made by districts such as returning 
to-be-scored answer documents to the wrong site or miscoding a school number on a document. 

2. The reply should include a description of where the answer documents will be scanned, the types of 
scanners to be used and their capacity during the time this project will be scored, general editing and 
proofing procedures of documents scanned, the types of documents and procedures the contractor 
proposes to ensure that the correct number of answer documents have been received and scanned to be 
scored. In the past districts packaged to-be-scored answer documents with a Document Count Form 
(DCF header) on top of documents of the same type. Alternate methods to ensure that the correct number 
of answer documents is scanned from each school may be proposed. See Section 7.4.1 for more 
information on the return of materials. 

3. The contractor will provide the Department with scan files for the PBTs and the extract files from the CBT 
system prior to creating the scored files. The contractor will draft layouts for these scan and extract files 
and incorporate changes as requested by the Department. The Department will specify the fields in the 
file, valid values for the fields, and the contents of each file (e.g., by grade or by subject). 

4. The contractor will provide two versions of each scan file. The original scan file contains the exact 
information as scanned and the second, the edited scan file, contains the final information from the 
scanning system incorporating any edits made as described in the Scoring and Reporting Specifications. 
For example, if the student wrote her name on the top of the answer document, but did not grid the 
bubbles on the document, the original scan file would have a blank in the name field and the edited scan 
file would have the name as entered by the contractor based on the hand-written name. The edit rules 
are designed to ensure a student will receive the correct report during the initial wave of reporting. The 
reply should include a description of how the edits would be made to original scan file data to ensure 
accurate and timely reporting of results. If necessary, the contractor will update the edited scan files to 
ensure the correct information is transferred to the first version of the State Student Results file (SSR). 

5. If any pages with responses cannot be scanned, all responses key entered into the system must be 
checked by a qualified person other than the person who initially key entered the responses. 

6. The contractor will maintain edit logs that indicate why records on scan files were edited and by whom. 
The logs will indicate the names of anyone who key entered or verified responses. 

7. The contractor will provide the original scan files and the edited scan files before scored files are created. 
See Section 7.5.6 for more information on the steps for verifying data and file delivery. If any edits were 
not made correctly, the contractor will update the edit files. 

See Section 7.6.2.2 for information on the records that must be scanned or extracted prior to the 
beginning of the psychometric tasks. The contractor's psychometric and scoring/reporting staff must 
coordinate schedules and activities to ensure that all files are transferred as expected (times, names, 
layouts, etc.). 
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7.5.4 Scoring 

The basic types of scored files created by the contractor and history files used and maintained by the 
contractor are described in this section. The contractor drafts detailed layouts for each file and then 
incorporates changes requested by the Department. 

7.5.4.1 Types of Scored Files 

From the PBT edited scan files and the CBT extract files, scored files are created. Scored files that 
contain data for students throughout the state are called State Student Results (SSR) files. These 
files contain information from the scan and extract files, information from the students' pre- 
identification labels when used, students' scores, and additional information regarding item (test 
question) data and scoring. The districts are also provided a truncated version of the SSR with just 
their students' data, but no item data. This file is called the District Student Results (DSR) file. 

1. Before the full-state SSRs are prepared, the psychometric staff (Department and contractor), 
begin their calibration and equating activities with a subset of the state records. These files 
are called calibration SSRs. Initially they only contain the raw score data, then after the 
calibration and equating, they are completed with the other derived scores as specified by the 
layout. See Section 7.6.0 for more information on the calibration, equating, and scaling 
procedures. Coordination of the details for the transfer of these files from scoring/reporting 
staff and psychometric staff is crucial to the timely release of reports and files. 

2. Aggregated files contain school, district, and state data based on the scores reported on the 
SSRs. These files contain information that is reported to the schools and districts as well as 
additional information such as standard deviations of the scores. The files with all the data are 
called State Aggregated Results (SAR) files, and the districts receive a District Aggregated 
Results (DAR) with the data for the state and their schools and district. 

3. After the initial release of files and reports, there are up to three waves of late reporting for 
students whose scores were not included in the original release of data, whose records had to 
be updated, or whose scores are released after suppression. The contractor will generate late 
reporting SSRs, DSRs, and provide student reports in a manner similar to the initial wave of 
reporting. SARs and DARs are not created for late reporting. 

4. After the last wave of late reporting, the contractor creates and provides close-out SSRs and 
DSRs that contain the final information and scores for that administration. 

5. After each summer EOC administration, the contractor will create annual SSRs, DSRs, 
SARs, and DARs by concatenating the SSRs and aggregating data combined for the fall, 
winter, spring, and summer administrations for the school year for subject tests that require 
multiple test administrations within a school year. The Department will specify how to handle 
any anomalies in reporting across administrations (e.g., a school name changed). 

6. For each spring close-out SSR and each annual SSR, the Department inserts demographic 
information from its Survey 3 membership file of student data and aggregates the data for 
approximately 50 demographic groups. The Department will provide these Demographic 
SSRs and SARs to the contractor. From these files, the contractor will create the 
Demographic DSRs and DARs and post these files to the online management system for 
district-level access. 

7. The contractor posts most files to the online management system, but may also be required to 
provide some files to the districts on CD/DVDs. See Table 7.5.5 for additional information. 

7.5.4.2 Types of History Files 

Three types of history files may be used for this project. The Department will provide the 
contractor with the layouts and necessary history files to begin the project, and then the contractor 
will maintain the files and update the files and layouts as instructed by the Department. 
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1. One type of history file may be used for the ELA/L Retake. Depending on policy decisions 
regarding scoring, the ELA/L Retake history file may be called the Alternate Passing Score 

(APS) file and would provide information on students who were grandfathered into a lower 
passing scale score requirement. Students who take the first administration of the Grade 10 
ELA/L before standards are set may be grandfathered into a lower passing cut score than set 
for other students. If this occurs, for each administration, the contractor will use this file to 
determine if a student is eligible for the APS and will update the records for students who test. 
The Scoring and Reporting Specifications will detail the matching criteria. 

2. The second type of history file will be used for the Algebra 1, Geometry, and Algebra 2 EOC 
Assessments for FDOE verification of the testing status (Retaker vs. First-time Tester) of each 
participating student. This kind of history file provides information for students beginning with 
the first time the subject test is administered statewide. Each administration, the contractor 
will use these files to determine if a student is a first-time tester or a retaker. The Scoring and 
Reporting Specifications will detail the matching criteria and process for using these files. 
Some aggregated data are based on first-time testers only, so the accuracy of the first-time 
status for each student is very important. 

3. A third type of history file will be used to track scores across grade levels for the G rades 3-11 
ELA/L and Grades 3-8 Mathematics students. Starting with the first ELA/L and Mathematics 
administration, the contractor will begin maintaining a file with student data for Grades 3-10 
ELA/L and Grades 3 - 8 Mathematics, called the ELA/L and Mathematics History File. 
Scores for up to 7 administrations will be added to this file using the closeout files for each 
administration. The contractor will update the file after each administration and will provide it 
to FDOE at the end of the contract. The Scoring and Reporting Specifications will detail the 
matching criteria and process for using these files. 

The contractor will update these history files each administration and provide a copy to the 
Department. 

7.5.4.3 Types of Scores and Codes on the SSR/DSRs 

1. The scoring process begins with the Test Maps that are created by the contractor when a test 
form is created. The Test Maps are used to identify item placement in the test book and other 
information about the item. The Test Maps include information regarding which items are core 
items and field test items, which responses are correct, and the category used for subscores 
within each content area. 

2. If the Test Map or a core item is determined to have a flaw, the Department may choose not to 
include an item in the students' scores. The contractor must have procedures to provide the 
flexibility to edit the Test Map information during the scoring process, if necessary. 

3. Excluding field test item data, subject raw scores and raw subscores are then generated by 
the contractor. 

4. See Section 7.6.1 for information regarding the calculation of scale scores (SS) by the 
contractor. 

5. Once the SS are determined, the score is classified in one of the Achievement Levels (AL) 
by the contractor. 

6. Florida statute requires that passing scores be set for all grades and subjects. The contractor 
will report the scores as specified by the Department. 

7. The ELA/L Retake may have two (2) cut scores for passing, depending on policy decisions 
that will be made prior to spring 2014. If so, the contractor will first determine if the student is 
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on the APS file described in this section and then determine the passing status for each 
student. 

8. For each special school, there is a defined School Type number that is used to determine 
which reports are prepared for a school and whether the results are included in district and 
state aggregated data. For example, home schools are school type 10, and they are not 
included in aggregated results within a district. 

9. Scores are not reported for some answer documents (e.g., students who did not answer 
enough questions to have a score reported). The student Score Status Flag determines 
whether the student scores are reported and aggregated. The Scoring and Reporting 
Specifications provide details on these codes. 

10. Many codes are related to each other and all values must be internally consistent. For 
example, the School Type Codes and the ESE Primary Exceptionality codes are used to 
determine the Curriculum Group and the Standard Curriculum Codes. The Ethnicity 
Code and the Race Codes are used to generate the Report Race Code. Students who use 
large print and Braille documents must have specific Primary Exceptionality Codes and 
accommodation codes that must be checked and updated by the contractor if necessary. 

11. Numerous other codes (approximately 15-20 for each subject) are provided on the SSRs. 
These types of codes may change occasionally based on changes in the assessment program 
and needs of the schools and districts. The contractor must be flexible in modifying the codes 
and including additional codes as needed to aid in the accurate and timely reporting of useful 
results. 

12. After the scores and codes are verified by the contractor, the first version of the full-state State 
Student Results (SSR) file is provided to the Department. The contractor will verify that the 
data were properly transferred from the corresponding edited scan file, that all scores, flags, 
and codes were set correctly, and that all fields contain valid values as defined by the layout. 
See Section 7.5.6 for additional information on the steps for verifying mock data, early district 
return data, and state data. 

7.5.4.4 Handling Special Print Documents 

1. This project includes Large Print, Braille, and One-ltem-Per-Page (LP, BRL, and OIPP) 

special print test materials. The contractor is responsible for numerous tasks to prepare the 
special print documents for scoring and reporting. The scores for students using these 
materials are included in the initial release of reports. 

2. The Scoring and Reporting Specifications will include the steps the contractor follows from the 
time these materials are unpacked until the scores are reported. If any to-be-scored answer 
documents are not packaged correctly, the contractor is responsible for properly preparing the 
materials or contacting the district if necessary. 

3. The contractor is responsible for the transcription of the responses from the LP book to the 
regular print book according to guidelines provided by the Department. The contractor may 
suggest ways to complete this process using the CBT system to expedite the scoring process. 

4. The contractor is responsible for transcribing Braille answer documents to regular print or into 
the CBT system. If a student uses an OIPP document, the contractor is responsible for the 
transcription of the responses from the OIPP book to the regular print book according to 
guidelines provided by the Department. Transcription services may be subcontracted, but all 
transcriptions must be completed in time to report the students' results in the initial reporting. 
This process must be carefully monitored and transcription must be double checked by a 
person other than the person who initially transcribed the responses. 
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5. The contractor will create a Special Document Tracking Log to inventory and track the 
processing of all special print documents. The contractor and the Department will determine 
the format of the log to track the answer document information and times for the log to be 
provided to the Department. 

7.5.4.5 Handscoring 

Handscoring refers to the processes necessary for determining the rating of a student's response 
on the writing prompts or on the performance-based tasks in ELA/L and Mathematics that 
respondents may propose. Both paper-based and computer-based tests will contain performance 
tasks, and so the Contractor must have the capability to score both hand-written and key-entered 
student responses. The Department requires handscoring processes that are reliable and valid, as 
well as efficient in terms of time and expenditures. This section describes procedural, design, and 
implementation requirements for scoring student writing and performance tasks. Respondents 
may propose other processes and procedures that accomplish the tasks in an equally reliable and 
valid manner, and are at least as efficient as those outlined below. The final processes will be 
agreed upon during the negotiation phase. See Appendix J for more details regarding 
handscoring. 

1. Students' constructed responses should be scored by trained readers using imaging 
technology. Respondents should provide a detailed explanation of the system that will be 
used. 

2. Respondents should also provide a detailed description of how the security of the test items 
and student responses will be maintained throughout the handscoring process. 

3. Trained professional readers will score written answers using criteria established by the 
Department in coordination with the contractor. Readers must have a college degree and 
must qualify to score Florida student responses by accurately scoring a series of training and 
qualifying papers. The readers must be monitored carefully and continuously throughout 
scoring for their accuracy (validity) and reliability (inter-rater agreement). In addition, team 
leaders must "read behind" each reader to determine the accuracy of their scoring. Readers 
must also be periodically given papers that have been previously scored to determine the 
accuracy of their scoring (validity responses). Quality control metrics will be established by the 
Department after contract execution. Student responses for constructed-response items in 
ELA/L tests in grades 3 and 10 and for Algebra 1 will be scored by at least two scorers (100% 
second read). For all other ELA/L and Mathematics performance tasks, responses will be 
scored by one reader, with twenty percent (20%) of these papers having a second reader. For 
these papers that are read a second time, if the two (2) readers' scores do not match, a formal 
resolution process will be established in order to obtain exact or adjacent scores for quality 
control purposes only. 

4. Handscoring may be done at the contractor's established scoring site(s). Respondents may 
also propose solutions that include distributed scoring. The scoring sites must be secure 
facilities that employ procedures and technology to control access and maintain the security of 
materials. Access to the sites must be limited to full and part-time employees and authorized 
visitors. Secure materials must be accounted for, and computers containing secure materials 
must be secured from unauthorized access or copying of files. 

5. The contractor should incorporate the following requirements into written handscoring 
specifications prior to scoring. The handscoring specifications will be a detailed guide to 
conducting handscoring that will be used by the contractor's handscoring managers and the 
Department. Draft handscoring specifications must be delivered by the contractor for review 
and approval at least three (3) months before the first test administration. The handscoring 
specifications should include the following: 
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a. Handscoring schedules. 

b. Site requirements (includes security and access during scoring). 

c. Scoring design (includes expected number of reads; groupings of items assigned to 
readers; assignment of first and second reads). 

d. Personnel (includes minimum requirements, such as education and experience, for 
scoring directors, team leaders, readers; number of personnel; security agreements). 

e. Rangefinding meetings (includes staffing, meeting procedures). 

f. Rangefinder review meetings (includes staffing; meeting procedures). 

g. Training and qualifying materials (includes listing/description of required materials; 
annotations for training and qualifying rounds; review and approval process). 

h. Validity sets (includes number and type of validity responses; transcription). 

i. Scorer degree verification procedures, 
j. Training procedures. 

k. Qualifying standards required of scorers before live scoring of student work. 

I. Handscoring reports (includes listing/description of handscoring reports; distribution). 

m. Scoring process (includes administration of validity sets; rules for determining and 
assigning resolution reads). 

n. Monitoring for scorer drift and other sources of scorer inconsistency and retraining 
(includes read-behind guidelines; use of daily and cumulative reader reports). 

o. Processing requirements (includes programming and design requirements for 
implementing online handscoring of images). 

p. Providing electronic copies of annotations used for training materials, calibration, and 
recalibration of readers, 

6. Department staff must be provided on-line access to all handscoring systems and reports and 
will communicate frequently with the contractor throughout the scoring process. During live 
scoring daily handscoring status calls must be held between the Department and the 
contractor. 

7. At the end of handscoring, the contractor must produce a report that captures at least the 
following results for all prompts and performance-based items and tasks that required 
handscoring: 

a. Item or task ID, the form number, and sequence number 

b. Total number of reads for each item 

c. Inter-Rater Reliability statistics including number and percentage of perfect agreement, 
number and percentage of adjacent agreement, and number and percentage of non- 
adjacent agreement 

d. Validity Statistics including the number of validity reads, number and percentage of 
perfect agreement, number and percentage of adjacent agreement, and number and 
percentage of non-adjacent agreement 

e. Distribution for scores and nonscorable codes from handscoring 
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f. The mean, median, modal score, and score variance and standard deviations for scorable 
responses 

g. Final quality-assurance monitoring reports from the handscoring system that include 
rescoring and retraining rates 

h. Scoring notes handscoring. 

The handscoring statistical data will be provided in an electronic data file in a format 
agreed upon by the contractor and the Department. 

Provide copies of all materials used in training and recalibration. 

8. It is expected that Department staff and Florida educators will be involved in the process to 
select student responses that will be used in training and qualifying scorers. 

7.5.5 Reporting 

This section includes requirements for reporting results to students, schools, and districts. This includes the 
posting of the scored files (DSRs, SARs, and DARs) described in Section 7.5.4, the release of hard copy 
student reports and PDFs of other reports described in this section, and the creation of files produced for the 
Department's website. 

Although the reports used in the past may be used as a model for these reports, the Department would 
consider changes to the report design to be more attractive or to present the data more clearly. The reply 
should include a description of how the respondent will design the reports, sample reports, and any 
suggestions for improvements to the current models. The report design is reviewed by the Department after 
each administration. If requested, the selected contractor will make modifications to the design after an 
administration and prior to the next administration. In addition, if high-level changes (change in Commissioner 
of Education, significant policy change) occur after designs have been approved; the contractor will make the 
appropriate modifications to the report design. 

If the respondent does not have qualified, professional staff devoted primarily to visual graphic arts and report 
design, a subcontractor(s) should be identified to provide these design services. The reply should include the 
names and documentation for the qualified staff or the subcontractors who will provide these design services. 

The final report design will take numerous iterations to ensure that all types of scores have been clearly 
presented. The Department will determine the final colors, design, and content of all reports. 

A general description of each type of report is provided below. Most of the reports are designed by test, that 
is, one (1) report form for English Language Arts/Literacy, one (1) for Mathematics, one (1) for each EOC 
subject test, and one (1) for Retake. 

Student Reports 

• For each student that tested by test. 

• Includes the student's scores (e.g., scale scores, achievement levels, raw subscores, passing status), 
information about the scores (e.g., passing scores, points possible), and comparison data (e.g., state 
means). 

• Two (2) hard copies (one (1) for parent and one for school) shipped to districts, packaged by school. 

• P rinted front and back with up to eight (8) colors plus black. 

• Some text translated to Spanish and Haitian Creole. The Spanish and Haitian Creole translations on the 
student reports must be created by someone familiar with the terminology used in Florida's schools and 
proofed by an independent party, that is, someone not involved in the original translation. 

• Delivered with cover memoranda and packing lists following specifications similar to those for distributing 
test materials. 

• Hard copies produced for initial reporting and late reporting waves. 
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• For E LA/L and Mathematics test, the student report will include historical scores that are up to seven (7) 
test administrations with a bar graph. 

• At all grades/subjects containing performance tasks, each student will receive a report showing his/her 
response to one performance task in each grade/subject. The report will include a copy of the answer 
given, the number of points earned on the item, descriptive information about the question, and scores 
(points earned) earned on all of the performance items. 

School Report of Students 

• For each school by subject test. 

• Includes the scores for all students in the school that tested. 

• One (1) PDF by school posted on the online management system for school access for initial reporting 
only. 

• One (1) PDF of all schools in a district posted in the online management system for district access for 
initial reporting only. 

• One (1) color plus black. 

• PDFs produced for initial reporting and late reporting wave. 

• Reports created for the late reporting wave include only the students with reports in that wave. The 
reports will denote that they are part of late reporting and the reports will be delivered to districts on 
CD/DVD. 

District Report of Schools 

• For each district by test. 

• Includes aggregated data for the schools in the district (mean scores, percent passing, etc.). 

• One (1) PDF by test by district posted in the online management system for school and district access. 

• One (1) color plus black. 

• One (1) Excel spreadsheet for all districts and schools by test with the same data as the PDFs for the 
Department to use for its web postings. This may be a slightly modified version of the P DF without color. 

• PDFs produced for initial reporting only. 

State Report of Districts 

• For each test. 

• Includes aggregated data for the districts in the state. 

• One (1) PDF by test, posted in the online management system for school and district access. 

• One (1) color plus black. 

• One (1) Excel spreadsheet by test with the same data as the PDF for the Department to use for its web 
postings. This may be a slightly modified version of the PDF without color. 

• PDFs produced for initial reporting only. 
District Summary 

• For each subject. 

• Includes the aggregated data for each grade level tested in the district and the state. 

• One (1) PDF by subject by district posted in the online management system for school and district access. 

• One (1) color plus black. 

• PDFs produced for initial reporting only. 
State Summary 

• For each subject. 

• Includes the aggregated data for each grade level tested in the state. 
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• One (1) PDF by subject posted in the online management system for school and district access. 

• One (1) color plus black. 

• PDFs produced for initial reporting only. 
CD/DVD/J ump Drive with Student Report Shipments 

• The initial shipment of Student Reports includes a CD, DVD, or jump drive with the DSRs and DARs and 
their layouts. 

• The late reporting shipments include a CD, DVD, or jump drive with the DSRs and layout and the PDFs of 
the School Reports of Students for the late reporting students. 

The Scoring and Reporting Specifications provide specific details on handling special student score flags and 
school types and the specific information and scores provided on each report. 

The packaging and distribution requirements for test materials (see Section 7.4.1) also pertain to the packaging 
and distribution of the reports. 

Table 7.5.5 below shows these reports grouped as they are released to districts and schools. 

Table 7.5.5 Results Released to Districts and Schools 



Table 7.5.5 Results Released to Districts and Schools 


Results 
Release 
Number 


Released To 


Method of Release 


Contents of Release 


1 


To Districts 


Online Management 
System 


District Student Results File (DSR .txt) 

School Report of Students (PDF of all students in district) 


2 


To Districts 


Online Management 
System 


State Summary (PDF) 

District Summary (PDF) 

State Report of Districts (PDF) 

District Report of Schools (PDF) 

State Aggregated Results File (SAR .txt) 

District Aggregated Results File (DAR .txt) 


3 


To Schools 


Online Management 
System 


State Summary (PDF) 

District Summary (PDF) 

State Report of Districts (PDF) 

District Report of Schools (PDF) 

School Reports of Students (PDF of own school only) 


4 


To Districts and 
Schools 


Shipments to 
Districts 


Initial Release Student Reports (hard copies) 
CD/DVD with DSR, SAR, DAR and Layouts 


5 


To Districts and 
Schools 


Waves of Shipments 
to Districts 


Late Reporting Student Reports (hard copies) 

Late Reporting School Reports of Students (PDFs on 
CD/DVD) 

District Student Results Files (DSR .txt) 


6 


To Districts and 
Schools 


Shipment to Districts 


Writing Images (PDFs on CD/DVD) (spring only) 


7 


To Districts 


Online Management 
System 


Demographic SARs, DARs, and DSRs 



Florida Standards Assessments, ITN 2014-47 



70 



7.5.5.1 Initial Reporting Guidelines 

The Department coordinates the schedule for the release of data to the public. The contractor must be 
flexible with the reporting release schedule. The general guidelines are provided in this section. 
Detailed guidelines are specified in the Scoring and Reporting Specifications. 

The basic steps for the initial reporting of full-state data for Department review are listed below. The 
Department may request slight modifications to these steps. 

a. The contractor posts the full-state scan and edit files version A for the Department on the 
SFTP site. 

b. If necessary, the Department provides an issue log for the state scan and edit files. 

c. The contractor posts the full-state SSR version A for the Department and any independent 
checkers specified by the Department (see Section 7.5.6). 

d. Contractor provides data and recommendations to the Department based on the review for 
anomalous scores; the Department will review and identify any data to hold for investigation. 

e. If necessary, the Department will provide an issue log forthe SSR files. 

f. The contractor posts the full-state SSR version B and the SAR version B (no version A) 

g. If necessary, the Department will provide an issue log. 

h. If necessary, the contractor corrects and reposts SSRs and SARs until approved. 

i. The contractor's psychometric and scoring/reporting staff verify the accuracy of the final files. 

j. As requested for initial checking, the contractor creates DSRs and DARs and posts on the 
SFTP site forthe Department review and updates as necessary until approved. 

k. During this process, the Department will identify a version of the files to be used to check the 
shipment of student reports and CD/DVDs. The contractor ships a sample of this shipment to 
the Department to check. 

I. If requested, the contractor posts additional samples of district and school PDFs of the reports 
on the SFTP site. 

m. After the PDFs are approved by the Department, the contractor will create all DSRs and DARs 
and reports on the SFTP site and, for state access only, in the online management system. 

n. After Department approval and guidance, the files and reports will be available for districts and 
schools in the online management system. 

o. The student reports and the CD/DVDs will be shipped for delivery by the critical date in the 
contract. 

7.5.5.2 Late Reporting Guidelines 

For each administration, various reasons make late reporting necessary. For example, to-be- 
scored answer documents were sent to the wrong location, updates were made based on the 
review of documents for graduating students (see Section 7.5.6.5), or the original student report 
was sent to the wrong school or student. 

Immediately after the initial wave of reporting, the contractor begins checking scores of graduating 
students as specified in Section 7.5.6.5. Any updates will be provided in the first wave of late 
reporting. 

The contractor will prepare for up to three waves of late reporting. The contractor and the 
Department check the late reporting files in a manner similar to the initial reporting wave. 
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Depending on the administration and number of inquiries from districts, the Department will 
determine the late reporting schedule. The contractor must be prepared to adhere to a rigid late 
reporting schedule to resolve any issues with reporting for one administration before the next 
administration begins. 

The late reporting shipments include the following reports and files. 

• Student Reports - two (2) hard copies as in initial reporting with a note or footnote denoting it 
is a late report. 

• CD/DVD with DSR and layout (no DAR) 

• CD/DVD of School Report of Students (not in the online management system) with a note or 
footnote denoting it is a late report. 

After the initial release of data, the Department opens a website called the Score Inquiry System 
for district staff to inquire about missing reports or to ask for a student's score to be reviewed. This 
system is designed so the contractor and the Department can communicate requests to search for 
a student's answer document or report, or to review a student's score. Maintenance of this system 
is the responsibility of the Department. 

The contractor must search for missing answer documents in the not-to-be scored materials if 
requested by the district to find any documents that were mislabeled or returned incorrectly. 

7.5.5.3 Close-Outand Annual Reporting Guidelines 

After the last wave of late reporting, the contactor creates close-out SSRs for each administration. 
These SSRs contain the final records for all student with erroneous records removed. For 
example, if the district/school number was incorrect in the initial reporting and corrected in late 
reporting, the close-out SSR contains the corrected record. The incorrect record is removed. 

The close-out files are provided to the Department for review and approval, but they are not posted 
for districts. 

After the summer EOC close-out files have been approved, the contractor creates annual SSRs 
following guidelines provided by the Department for tests that require multiple administrations 
within a school year. These annual SSRs contain the records for students from each 
administration that school year. A few fields may be slightly altered from the close-out files for 
each administration, for example, if a school name changes during the school year, the name will 
be updated so the SAR shows annual aggregated data for that school using the correct name. 

7.5.5.4 Demographic Reporting Guidelines 

After the spring close-out SSRs are approved by the Department, the Department inserts 
demographic information from its Survey 3 membership file of student data and aggregates these 
data for approximately 50 demographic groups. The Department then returns the Demographic 
SSR and SAR to the contractor to create Demographic DSRs and DARs. 

After the Demographic SARs are created and approved, the contractor also creates Excel 
spreadsheets with the data that can be used for the Department's interactive demographic 
websites, which can be viewed here for reference: 

https://appl.fldoe.org/fcatdemoqraphics/ and 

http://appl.fldoe.org/FEocDemoqraphics/MainMenu.aspx . 

The Department will specify the format for the spreadsheets. 

After the annual SSRs and SARs are created for the subjects that require the multiple test 
administrations within a school year, this same process is completed for those files. 
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7.5.6 Verifying Data 

Due to the tight timelines and quantity of data, all initial scoring and reporting processes are checked with 
mock data prior to the administration. Then, once actual data are available, a small sample of answer 
documents are used to begin checking, called early district return data. After all answer documents have 
been scanned, the files and reports are checked for completeness, accuracy, and possible irregularities 
(cheating) as described in this section. The verification process may require numerous iterations of the files to 
ensure that each student's scores are accurately reported. 

The reply should include a description of the procedures to be used to verify the accuracy of data as outlined 
in this section. This includes the staffing necessary to complete the mock data process before the 
administration begins and the verification of each version of the files given the tight time constraints. 

The Department may have independent contractors assist in a review of the procedures used in this project 
and verification of the files and reports. Currently the Department has contracts with two independent parties 
for this purpose. One (1) contractor is primarily concerned with reviewing the procedures and the 
specifications for the scanning, scoring, and reporting with an emphasis on the psychometric aspects and 
primarily coordinating with the psychometric staff. The other contractor is primarily concerned with ensuring 
the procedures are followed, verifying codes and scores on the scan files, the SSRs/DSRs, the SAR/DARs, 
the Student Reports, and the PDFs of the other reports coordinating with the scoring and reporting staff. 

The contractor will be responsible for coordinating with the Department's independent contractors for the 
review. This may include touring the scanning site and discussions with the contractor's scanning, scoring, 
psychometric, and reporting staff. During the psychometric activities, the independent contractors may 
participate in the meetings and discussions, and during the scoring activities, the contractor may be asked to 
provide a portion of the reports and files to the independent contractors when reports and files are provided to 
the Department. 

7.5.6.1 Checking Mock Data 

1. All mock data files, reports, and procedures are generated and delivered using methods as 
close as possible to the same way final files will be generated and delivered. This process 
begins with mock PrelD files, mock PrelD labels, mock PrelD Frequency Reports, mock 
history files, and mock answer documents, and then all headers, files, and reports for the initial 
wave of reporting are generated using the mock data. The mock student reports are packed 
and shipped as specified by the Department with the media containing the DSR and DAR, and 
the mock files and PDFs of reports are also posted to the SFTP site for the Department 
review. 

2. Actual school information from the MS ID file will be used in the mock data process to check 
the processing of the special school types and special districts. 

3. Mock data cases may be generated using different methods. To check scanning of PBTs, 
some mock cases must be manually gridded, but others may be auto-gridded (preslugged) 
documents. Some CBT data must be generated by the contractor staff simulating the 
processes thata CBT student would use for each form of each test. To have enough cases 
to check all scoring and aggregating rules without using too many paper answer documents 
and time entering CBT responses, some additional cases may be computer generated, 
meaning for PBTs physical documents are never gridded or scanned for those cases and, for 
CBTs no one sat at the computer and key entered the responses in the same manner as a 
student enters the responses. 

4. For PBTs, only the cases to check erasures must be manually gridded and other references in 
this section to cases with gridding may be auto-gridded. 
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5. For each CBT, the PBT accommodated form must also be checked. Large Print and Braille 
paper documents are not used for mock processing; however the regular print answer 
documents and the headers are created with the appropriate Large Print and Braille codes. 

6. The Scoring and Reporting Specifications indicate the various cases that must be included in 
the mock data. To easily check that all values scanned on PBTs and key entered on CBTs 
are properly recorded on the files, the mock data will include cases to check the values using 
specific patterns (e.g., alternating A,B,C,D,A,B,C,D responses). Cases must be included to 
be sure that each value on the SSR, SARs, and reports are accurate. This includes 
processing erasures, multigrids, demographic values, score flag codes, school type values, 
scores, etc. 

7. The contractor must design the mock data files to check all procedures through the release of 
results. To reduce the amount of time to review these data and iterations required, the 
contractor will complete their review of all mock cases, possible values of codes and scores, 
and all reports before providing the data to the Department. The reply should include a 
description of how the respondent will complete these mock data, mock files, and mock 
reports p_rior to the first day of testing for each administration. 

7.5.6.2 Checking Early District Return Data 

1. To begin the verification of actual (live) data, the Department will identify some districts to be 
processed first, called Early Districts. The contractor will arrange for the pickup of answer 
documents from these districts so they will be scanned first for Department review. The CBT 
records for students in these districts will be extracted and provided to the Department for 
review. 

2. The selected contractor must anticipate that the spring scan check will be completed at the 
contractor's scanning site. The scan check for other administrations may be conducted at the 
contractor's scanning site or at the Department. If the scan checking is performed at the 
Department, the contractor will post PDFs on the SFTP site of the headers and answer 
documents or ship the required documents to the Department to complete the checking. 

3. For the on-site review of answer documents, the contractor will provide the Department the 
original scan and the edited scan files at least three (3) business days prior to the on-site 
review. 

4. The contractor will pay travel expenses for up to five (5) Department staff from Tallahassee to 
the contractor's scanning site to complete this checking. These expenses must be included in 
the reimbursable funds described in Section 7.7.9. 

5. During the on-site review, the contractor will provide computers to Department staff with the 
following: 

• Access to the CBTs, access to email, access to a high-speed printer, and, if requested, 
SAS. 

• Easy access to the answer documents. Batches and stacks will be logically sequenced 
with the headers used. 

• LP, Braille, and OIPP documents and updated Special DocumentTracking logs. 

• Key E ntry verification logs (e.g., double entry of responses if document is unscannable) 

6. The Department's independent contractors may participate in this review or they may visit the 
site prior to this review. 

7. Depending on the Department staff availability, the on-site review may be completed during 
one week or in two consecutive weeks. 
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8. The reply may include additional and alternate methods to aid in meeting the data verification 
and reporting requirements in the time allowed. 

7.5.6.3. Checking Data for Anomalous Scores 

Before data are released to the public, the contractor must review the student and school scores 
for anomalous data, that is, data that may not be a valid measure of student achievement. The 
contractor must review the data for testing irregularities, such as a student copying another 
student's answers or an administrator changing answers. 

If the respondent does not have staff to perform these analyses, a subcontractor should be 
proposed. The Department will approve the subcontractor and final procedures to be followed for 
this checking. 

Anomalous scores would include the following. 

• Students in the same testing group have similar response patterns 

• A student has an excessive number of erasures from wrong answers to right answers 

• Students in a testing group have an excessive number of erasures from wrong to right 
answers 

• There was an unusual increase in school performance from a comparable administration 

The guidelines and procedures used to perform the anomalous score checking will be refined by 
the Department with the contractor's assistance. 

If an irregularity is found in the data, the Department may request for the contractor to hold (i.e., not 
release) some data. This is handled using the Score Status Flags between the first and second 
versions of the full-state SSRs. Schedules must be set to allow time for this review and 
updating before the initial reporting for each administration. 

7.5.6.4. Checking Full-State Data - Initial Reporting 

1. The contractor will verify that all files and all PDFs of reports have been produced according to 
the Scoring and Reporting Specifications before providing them to the Department. In addition 
to the checks described in this section, the contractor's psychometric staff must also approve 
the final SSRs and SARs before final reports are generated and data are released to the 
public. 

2. The verification begins by checking that the DCFs have been completed properly so student 
reports will be sent to the correct school. If the school identified on the header is different from 
the school on the student's answer document, and both are active schools, the contractor 
must identify the correct school for the student by contacting the district, if necessary. 

3. The contractor will compare the edited scan files to the SSR and then the SSR to the SAR, 
and compare reports to the SSR and SAR to be sure all information (scores, codes, school 
numbers, etc.) is accurate and flowing through the systems correctly before submitting to the 
Department for review. The contractor will verify that all scores, flags, and codes were set 
correctly, and all fields contain valid values as defined by the layout. 

4. The contractor will compare the final number of students tested in each school to the number 
of students on the PrelD files to identify any anomalies, such as a missing box. To ensure all 
answer documents have been received for scoring prior to the delivery of the first scan file to 
the Department, the contractor will review the numbers of records and contact the Department 
and districts, using guidelines defined in the Scoring and Reporting Specifications, of any 
concerns. The contractor will assist districts in returning any materials that were not handled 
properly. 
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5. As specified by the Department, the contractor will also provide sample shipments including 
the student reports and the CD/DVDs with files simulating the shipments to districts. The 
Department will determine which districts shipments will be included in this verification to 
ensure that a variety of reports are checked. They may be different from the districts selected 
for early scan if those records are not diverse. As described earlier in this section, the 
contractor may also be requested to send some reports to the Department's independent 
contractor. 

6. Before the files and reports are available in the online management system, the contractor and 
the Department will verify that the correct reports and files are available for district and school 
users prior to the release of the data. The contractor will provide documentation to show that 
the correct files and reports are being released as specified in the reporting schedule. 

See Section 7.5.5 for more information on the reports. 

7.5.6.5 Checking Graduating Student Scores (Manual Rescore) 

Due to the high stakes of the tests for graduating students, the contractor must perform additional 
checks for graduating students who test on paper for Grade 10 ELA/L and for Algebra 1. The 
checks must be completed so any updates may be included in the first wave of late reporting. 

1. For the tests required for graduation, the contractor must identify student in each grade who 
failed by 5 or fewer Scale Score points or 15 Developmental Scale Score points for 

Manual Rescoring. If there is an APS cut score established for the assessment, the APS cut 
score must be used for selecting the students. 

2. For each of these students, the contractor will review each answer for the subject failed and 
verify that the items have been scored as the student intended. For example, the contractor 
will look for unintentional stray marks in the answer bubbles that do not appear to be an 
intentional grid, will confirm the hand written name on the document is the same as on the 
PrelD label if one (1) is used, and will also confirm that no incomplete erasures in the gridded- 
response columns were recorded as a mark (e.g., double gridding because of incomplete 
erasure). The item responses will be compared to the values on the SSR for any irregularities 
in editing or processing. 

3. If any editing, scanning, or processing irregularities are detected, or if an incomplete erasure 
would change an item response, the student's paper must be rescored with the updated 
response(s), and a new SS must be calculated to determine the passing status of the student. 
If the student passes, the record must be updated for the first wave of late reporting. If not, 
no change is made in the record. 

4. A new student report will be generated and shipped to the district and the student will be 
included on the School Report of Students sent with the other late reporting wave 1 student 
data, if any. 

5. The contractor will provide the Department with the number of answer documents reviewed by 
subject and the number that will have a Passing Status changed to Y when the new files are 
submitted. 

6. This process will be completed within sixty (60) days of the original release of scores for the 
administration. 

7.5.6.6 Checking Late Reports 

The contractor and the Department check the late files, reports, and CD/DVDs in a manner similar 
to the initial files and reports, although the files and reports produced are slightly different. The 
Department checks the files and reports after the contractor. 

See Section 7.5.5.2 for information on the procedures for the late reporting waves. 
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7.5.6.7 Checking Close-Outand Annual EOC Files 

The contractor checks the close-out files by comparing each to the files previously generated for 
the administration to ensure that records reported to the incorrect schools have been removed and 
any scores or information for students have been updated or added as necessary. 

The contractor will ensure that there are not erroneous duplicate records caused by the multiple 
waves of reporting on the close-out files. The contractor must review the Late Reporting issue logs 
to ensure erroneous records have been removed from these files. The Department checks the 
files and reports after the contractor. 

The contractor checks the annual EOC files similar to the initial files, ensuring that the first- 
time/retaker status codes are properly set. The Department checks the files and reports after the 
contractor. 

7.5.6.8 Checking Demographic Files 

Each spring the contractor will check the Demographic SSRs created by the Department to verify 
that no records or scores were changed; only Survey 3 data will have been inserted. The 
contractor will verify that only valid values are in the Survey 3 fields before creating the 
Demographic SARs. 

The contractor will check the Demographic DSRs, SARs, and DARs prior to release in a manner 
similar to the initial release of the files. The Demographic Excel spreadsheets must also be 
checked by the contractor to be sure they contain the same information as on the SAR. The 
Department checks the files and reports after the contractor. 

See Section 7.5.5.4 for information on the procedures for the demographic files. 

7.6.0 Scaling, Equating, Scoring, and Special Psychometric Studies 

During the timeframe of this ITN, the contractor will be responsible for drawing samples from the populations 
of students, for calibrating field-test items, for constructing the new test forms for the future administrations, 
and for Scaling, Equating, and Scoring (SES) of the assessment system. The system is complex and requires 
the contractor to provide knowledgeable and experienced staff, including psychometricians and programmers. 

Other than the annual regular psychometric operations, such as sampling, test construction, and SES for the 
assessment system, the contractor will conduct a set of special psychometric studies for these assessments, 
described in Section 7.6.3.6. The contractor will also cooperate to conduct another set of psychometric 
studies as Florida transitions from FCAT 2.0 and EOC Assessments to the Florida Standards Assessments 
(FSA) systems. As the Department transitions to the FSA, the contractor will assist the Department in the 
establishment of FCAT 2.0 and EOC equivalent score scales, appropriately calibrate items in the new 
assessments, properly combine performance indicators, construct meaningful score scales, possibly establish 
vertical scales for E LA/L and Mathematics assessments, assist in defining achievement level descriptors, build 
cut scores with criterion-referenced meaning on the new scales, link the performance of Florida's students to 
the scales of other assessments to make comparisons, and conduct other special analyses that may be 
required by the Department or required for a peer review under the federal Elementary Secondary Education 
Act (ESEA) program. This section of the ITN provides descriptions of related services and products and the 
expected quality assurance steps that must be employed to ensure that all tested students receive 
assessment results that are valid, reliable, and accurate. 

Florida uses a Technical Advisory Committee (TAC) composed of selected Florida District Coordinators of 
Assessment and selected experts from around the nation representing the areas of measurement and 
statistics, students with disabilities, English language learners, and related alternate assessments. The TAC 
members listen to issues and offer guidance on implementation of Florida's assessment program. Currently 
Florida's TAC meets twice a year - in March and in November. The contractor is responsible for convening 
and conducting these meetings. 
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The contractor may utilize proprietary software for any of the annual regular psychometric activities or the 
special psychometric studies for the assessment system during the transition to the FSA, but is required to 
provide this software for use by all relevant staff, including the Department and any subcontractors. Whatever 
software is selected must be able to handle the large number of test cases, including large calibration 
samples, and also accommodate full-state calibration with numbers of records in excess of 200,000 at a time. 

The contractor will be responsible for assembling a psychometric team to support work performed under this 
contract consisting of the contractor's psychometricians and programmers, and at least one (1) outside 
entity subcontracted to participate in all SES activities to independently perform these particular 
operations and verify all results. The Departments psychometric team consists of two psychometricians plus 
three graduate interns pursuing their doctorate in educational measurement to be part of the team replicating 
all SES activities and overseeing the entire process. 

A separate procurement is being issued for the FCAT 2.0 and Florida EOC Assessments for subjects not 
covered as part of this ITN (Science, Social Studies, etc.). The contractor selected as part of this ITN process 
should anticipate coordinating with the contractor from the FCAT 2.0/F lorida EOC Assessments procurement 
for some of the tasks described in this scope of work. 

7.6.1 Assessments 

7.6.1.1 Florida Standards Summative Assessments (Scaling, Equating, and Scoring) 

1. The contractor will develop and maintain test construction specifications describing the details 
of test construction, including content and psychometric guidelines for building equivalent 
forms. Item selection will be based on matching target test characteristics, test information 
functions, and standard error curves in addition to meeting other content and psychometric 
requirements, plus further constraints. 

2. For the scaling of items, the Department prefers to use a three-parameter logistic Item 
Response Theory (IRT) model for multiple choice items, two-parameter logistic IRT model for 
gridded-response items, and two-parameter generalized partial credit model for constructed- 
response items. For the equating of the tests to their base year scales, the Department 
prefers to employ test characteristic curve equating using the Stocking & Lord procedure and 
triangulating the equating with IRT based other equating procedures, such as Haebera, 
Mean/Mean, and Mean/Sigma. For scaling student performance, the Department prefers to 
use Maximum Likelihood Estimation to score examinee response patterns. For 
dimensionality, the Department supports using a variety of item types measuring various 
student skills on the Florida Standards, but maintains validity for a single dimension. However, 
these are just the Department's preferences for scaling, equating, scoring, and dimensionality. 
The contractor must conduct thorough research studies and collect empirical evidence to 
determine whether these options are psychometrically defensible, whether items measuring 
various constructs can be combined into a single assessment instrument and scored, and 
whether the preferred options can be implemented operationally to obtain assessment results 
that are valid, reliable, and accurate. If research and empirical evidence(s) suggest the use of 
a different set of techniques in scaling, equating, and scoring of the assessments, the 
respondent should provide a detailed description of the measurement models proposed for 
use in calibration, equating method for creating interchangeable scale scores, and scoring 
algorithm for placing examinees onto a score scale. The contractor will be responsible for 
designing sampling plans, drawing the samples from populations of examinees, and supplying 
the data files to the scaling and equating team described in Section 7.6.2.1 so that the 
required analyses can be performed in a timely manner. The contractor will be responsible for 
calibrating the tests, placing item parameter estimates onto the base year's scale, and 
providing the final scaled item parameters for the purpose of scoring. After the Department 
approves the final scaled item parameters, the contractor will score student responses, the 
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entity subcontracted to participate in all SES activities will independently verify the scale 
scores, and the Department's psychometric team will replicate and verify the scale scores. 

3. For the Assessments in Algebra 1, Algebra 2, and Geometry, the contractor must construct 
multiple forms of these assessments by following content and psychometric requirements as 
indicated in the test construction specifications. As for the scaling, equating, and scoring 
process for these assessments, the Department desires to follow the same process as 
indicated in the previous paragraph, but since there are multiple forms of the assessments at 
this time, the Department prefers to concurrently calibrate these forms so that all item 
parameter estimates can first be placed onto a common scale, and then, equating these item 
parameter estimates to the base year scale via anchor items. In other words, equivalency of 
the multiple forms of new tests is verified through a concurrent calibration, plus post-equating 
via common items between the item bank and the new test forms. However, as indicated 
earlier, the contractor must show evidence that the Department's preferences are 
psychometrically defensible and operationally feasible. The respondent may include in the 
reply a different proposal for scaling, equating, and scoring of these assessments to obtain 
assessment results that are valid, reliable, and accurate. Also, when multiple forms of these 
assessments are being created, the contractor must construct the new test forms that are 
sufficiently different in an effort to discourage cheating and to ensure a valid measure of 
student learning. In addition, the department will consider the use of pre-equated forms for 
administrations other than the spring administration. 

4. Because every effort is made to develop items and construct tests according to universal 
design principles, we anticipate all Braille versions of the assessments will contain the same 
items as the regular version of the tests. If they do not contain the same items, the contractor 
will develop a separate scoring process to be approved by the Department and used by the 
contractor and subcontractor. 

7.6.1.2 Retake Assessments (Scaling, Equating, and Scoring) 

The contractor's content and psychometric teams will work together with the Department's content 
and psychometric teams to select retake test forms, which may consist of a previously-used intact 
form, or another proposed design that yields a test of similar difficulty to the operational form. The 
inclusion of performance tasks in the retake form may be proposed, however, consideration will 
also be given to forms that do not include performance tasks. 

The contractor will administer the selected test form and will score student responses. The entity 
subcontracted to participate in all SES activities must independently verify the scale scores 
generated by the contractor. The same procedure will be repeated to select different forms for 
subsequentyears. 

7.6.2 Scaling, Equating, and Scoring (Psychometrics) 

1. The contractor will construct Scaling, Equating, and Scoring (SES) specifications for the assessments. 
The specifications will describe the specific steps for implementing the scaling, equating, and scoring 
requirements and will explain how, by when, and by whom all of the necessary steps will be completed 
and verified. The specifications will also address the steps, procedures, and software required for 
replication of the scaling, equating, and scoring procedures by the Department. The specifications will be 
reviewed by the Department and TAC members, if necessary, and updated by the contractor for each 
subsequent administration. The Department will provide the contractor with copies of existing 
specification documents. 

2. Before operational SES activities start, the contractor and other members of scaling and equating team 
must conduct dry-run SES activities on mock data files to make sure all operations laid out in the SES 
specifications can be successfully executed during the operational SES process. For this purpose, the 
contractor will prepare "mock" data for the assessments using the State Student Results file (SSR), as 
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indicated in Section 7.5.6.1. The purpose of this mock data will be to provide a vehicle for the SES dry 
run; i.e., a practice check of the operation of computer programs used in scaling, equating, creating 
adjusted item parameters, and scoring. The psychometric staff for the identified calibration team will be 
available for daily discussions and consultation throughout the parallel dry-run calibration activities. 

3. It is the responsibility of the contractor to keep all the documents/electronic files from the scaling, 
equating, and scoring activities, such as the classical and IRT item statistics, alternative equating 
solutions summarizing the reasons why anchor items are removed or kept in the anchor set, the impact of 
decisions on the transformation coefficients and scale scores, and the other pertinent decisions made by 
the Department. The format of these documents must be proposed by the contractor and mutually 
agreed upon by the contractor and the Department. 

4. The SES specifications must include how the contractor's subcontractor will independently verify the 
scaling, equating, and scoring operations. For the entire SES process, the contractor will provide all of the 
resources, including software, staff support, and data files to permit parallel calibration, equating, and scoring 
of data during the same time period that contractor are producing the scaled item parameter estimates. 

7.6.2.1 Scaling and Equating Team 

The contractor will assist the Department in coordinating the work of the scaling and equating team 
during dry-run activities and regular administrations each spring. The contractor is required to 
subcontract with at least one outside entity to work on the scaling and equating team during all 
SES activities. The contractor's, the subcontractor's, and the Department's psychometric staff will 
each independently perform SES tasks to verify results. The specifications will clearly describe the 
scaling and equating team consisting of the Department's, the contractor's, and third party's 
psychometricians and programmers, as well as the roles assigned to each entity. The 
Department's psychometric team will oversee the entire SES process. Respondents should 
indicate in their replies the qualified personnel/subcontractors who will provide these professional 
services and include their vitae or other documentation of expertise and experience. 

7.6.2.2 Selection of Calibration Samples 

Anchor, core, and field-test items are calibrated on representative samples. Thus, the contractor is 
responsible for designing sampling plans and drawing the representative samples from the 
respective populations, which will include at least 65% of the respective population, including 
approximately 90% of Miami-Dade. The contractor will also plan in a way so that a representative 
set of approximately 5,000 students take anchor and field-tests forms. The contractor's sampling 
specifications must include analyzing the most recent SSR files and establishing a road map for 
gathering the required data proven to represent the demographic characteristics of the state and 
student performance. The specifications will also address the steps, procedures, and software 
required for replication of the sampling procedures by the Department. The specifications will be 
reviewed by the Department and TAC members, if necessary, and updated by the contractor for 
each subsequent administration. 

Before SES activities start, the contractor must also check and verify the representativeness of the 
most recent data, and then provide the files to scaling and equating team members so that 
necessary analyses can be performed in a timely fashion. 

The contractor must repeat this process for each assessment administered each spring. 
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7.6.3 Technical Oversight and Special Studies 

7.6.3.1 Contractor's Role 

The contractor must provide psychometric direction and oversight for all aspects of the assessment 
programs described in this ITN. The psychometricians assigned to this project must have 
extensive experience in state assessment programs. The psychometricians will plan, conduct, 
analyze, and report on test construction, scaling, equating, scoring, and the special studies 
described in this section. 

7.6.3.2 Technical Report 

Each year, the contractor will produce a technical report based on an overall analysis of that year's 
administrations. The technical report will also include a comparison of the characteristics of the 
current test administration to previous administrations. The technical report will be reviewed by the 
Department and by the Technical Advisory Committee (TAC) prior to completion of the final copy of 
the report. The technical report will include a section on field-test administrations. The report will 
include tabular and graphic displays of data to illustrate the characteristics and quality of test 
scores. At a minimum, the technical report will include the topics listed below in the draft table of 
contents. 

A. DRAFT TECHNICAL REPORT TABLE OF CONTENTS 

B. BACKGROUND OF ASSESSMENTS 

C. ITEM DEVELOPMENT & TEST CONSTRUCTION 

• Item Development 

• Item Review 

• Field Testing 

• Item Statistics 

o Classical Statistics 

o Item Response Theory Statistics: Difficulty, Discrimination, and Guessing 

o Differential Item Functioning (DIF) Statistics 

o Model Fit 

o Local Independence 

o Dimensionality (factor analyses, etc.) 

• Test Construction 

D. ADMINISTRATION 

• Eligibility 

• Administration Procedures 

• Accommodations 

• Test Security 

• Data Forensics Program 

E. REPORTS 

• Appropriate Uses for Scores and Reports 

• Reports Provided 

• Description of Scores 

• Appropriate Score Uses 

• Cautions for Score Use 

F. PERFORMANCE STANDARDS 

• Setting Performance Standards 

• Interim Performance Standards 
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G. SAMPLING 

• Rationale 

• Procedure 

• Representativeness 

H. SCALING 

• Rationale 

• Measurement Models 

• Scale Scores 

• Scale Drift 

• Vertical Scaling 

• Checking IRT Assumptions (Model Fit, Local Dependence, Dimensionality, etc.) 

I. EQUATING 

• Rationale 

• P re-Equating 

• Post-Equating 

• Development Procedure for Future Forms 

• Sources of Error Variances to Score Instability 

• Linking Assessments During Transitioning Years 

• Linking Assessments to Other Scales for Performance Comparision 

j. RELIABILITY 

• Estimating Reliability 

• Standard Error of Measurement 

• Student Classification Accuracy and Consistency 

• Reliability for sub groups in the population 

• Reliability of year-to-year changes in school means 

K. VALIDITY 

• Perspectives on Test Validity 

• Criterion Validity 

• Content and Curricular Validity 

• Construct Validity 

• Consequential Validity 

L. CONSTRUCTED-RESPONSE SCORING 

• Scoring Process 

• Quality Control 

• Handscoring Reliability and Validity 

• Rater Effects 

M. QUALITY CONTROL PROCEDURES 

• Quality Control for Test Construction 

• Quality Control for Non-Scannable Documents 

• Quality Control in Data Preparation 

• Quality Control in Production Control 

• Quality Control in Scanning 

• Quality Control in Editing and Data Input 

• Quality Control forTest Form Equating 

• Quality Control in Scoring and Reporting 
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N. SUMMARY STATISTICS 

• Item level statistics 

• Test level statistics 

• Review and comparision of raw scores, sub-scores, scale scores, distributions for total 
and sub-populations across years to monitor the trends across years and sub-populations 

• Review and comparision of percent proficiency rates across years on different populations 

• Frequency distributions of student achievement (all students and by sub-group) across 
years by grades and subject 

• Scale drift indicators 

• Correlations of student performance across the various subject areas tested 

Following each field test or operational administration, differential item functioning (DIF) analyses, 
to detect possible item bias, will be conducted, preferably using Mantzel-Haenszel method. DIF 
analyses are conducted for Caucasian, African-American, and Hispanic racial/ethnic groups and by 
gender. Values for items resulting from these analyses will be included in the item-banking 
system. Changes in DIF values across administrations will be analyzed and presented in the 
Technical Report. 

The contractor will provide a detailed item analysis for each test administration and provide the 
results in the annual Technical Report. There are several item level statistics, such as point- 
biserial, corrected point-biserial, fit, etc. available in the current item bank. Thus, the contractor is 
responsible for calculating these statistics following each administration and populating the findings 
into their respective positions in the bank. 

The Department will work with the contractor to determine the contents of the written report. The 
contractor will provide several printed copies of the report and also an electronic version in the 
format determined by the Department. 

7.6.3.3 Transitioning from FCAT 2.0 and EOC Assessments to Florida Standards Assessments 
(FSA) - Linking the Scales 

As the assessment system transitions to the FSA in spring 2015, the scales of the new FSA will not 
yet have any meaning attached. Therefore, the Department requires technical assistance from the 
contractor so that the scales of these assessment systems can be linked through a special and 
defensible psychometric operation. The contractor will provide a detailed scale linking plan for 
transitioning from FCAT 2.0 Reading & Mathematics and EOC Algebra 1 & Geometry to FSA. 

Linking through a special psychometric operation may go beyond employing a simple linking 
procedure, since each score unit on the two separate scales may have to be linked to each other 
so that achievement levels can be determined for student and school accountability purposes. The 
Department may directly engage additional contractors, or the contractor's subcontractor (if 
proposed) to triangulate the results. 

The plan will include alternative options for linking, how the alternatives will be operationalized, 
scoring plans, and procedures. The plan should include a comprehensive and detailed timeline for 
all tasks and events. The plan will be reviewed and vetted by Florida TAC members after 
submission, and may require revision as a result of this review. This review will address the 
appropriateness and validity of the psychometric procedure selected for linking, as well as the 
validity, reliability and defensibility of the linked scores. 

The respondent's reply should describe available psychometric methods for establishing interim 
scores for high-stakes student-level decisions, including an explanation of the defensibility of the 
proposed methods. During the transition years, as student performance on other assessments will 
be used to determine the achievement of testing standards, the linking/concordant score studies 
will be conducted by the contractor and concordant scores will be recommended to the 
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Department. The contractor selected through this ITN process will be expected to secure external 
legal opinion regarding the defensibility of the linking method to be employed. 

7.6.3.4 Implement Achievement Level Standard Setting Meetings 

The contractor is responsible for facilitating the Department's process to establish achievement 
levels and associated cutscores for the Florida Standards ELA/L, Mathematics, Algebra 1, Algebra 
2, and Geometry Assessments; achievement levels, passing scores, retention cut for grade 3 
ELA/L, graduation standards for ELA/L, and achievement levels and passing scores for the Algebra 
1, Algebra 2, and Geometry Assessments in consultation with Florida educators and citizens. 

The contractor will be responsible for developing a Standard Setting Plan for each standard-setting 
process utilizing IRT item values and appropriate, reliable, valid, and defensible procedures. The 
Standard Setting Plan will address the nature of the proficiency level standards, methods for 
determining the standards, and procedures for validating and analyzing the quality of information 
reported using the achievement levels. In addition to describing the general standard-setting 
strategy, the Standard Setting Plan will describe in detail how participants will be identified and how 
the standard-setting procedure will be implemented, such as the materials to be prepared by the 
contractor, how test forms will be organized, and how the contractor will analyze the data and 
present impact results. 

The contractor will be responsible for organizing and implementing the standard-setting processes, 
based on the Standard Setting Plans, and for assisting the Department in conducting the standard- 
setting meetings. Each standard-setting process will involve meetings of standard-setting 
committees, which will involve Florida teachers and instructional leaders, followed by a reactor 
meeting involving other Florida education stakeholders. The educator meetings in summer 2015 
will result in content-based achievement level descriptions for each grade and recommended cut 
scores for each test. The Department desires to present proposed cut scores to the State Board of 
Education in J anuary following each meeting; therefore the final report of activities of each meeting 
should be available for the Department no later than November following each meeting. The final 
report of standard setting activities should include in detail process and activities followed during 
the standard setting, including a description of materials used, statistical background, participants, 
changing of recommendations through rounds, and participant evaluation of meeting processes. 

7.6.3.5 Vertical Scaling 

The contractor will be responsible for conducting the vertical scaling for the Florida Standards 
ELA/L and Mathematics Assessments. This scale provides for reporting growth continuously from 
grades 3 to 10 in ELA/L and from grades 3 to 8 in Mathematics. In other words, each student's 
"growth" between adjacent grade levels will be calculated and reported on this scale as a type of 
"value-added" score. 

The contractor's plan for vertical scaling, provided prior to the selection of items and construction of 
forms for the appropriate administration, should recommend and describe the process of 
determining hierarchical common content areas across grades, a recommended method for 
vertical scaling that takes into account potential uses of the scale for NCLB and Florida's 
accountability system, the data collection procedures, and how and by whom and by when all of 
the necessary steps will be completed. The vertical scaling plan must be approved by the 
Department and presented to the Technical Advisory Committee. After vertical scaling studies are 
conducted, the contractor will prepare a technical report including a description of the process, 
statistical background, methodology, the recommended scale, and characteristics of the scale. 
The final report will be presented to the Technical Advisory Committee. 

The contractor should anticipate replicating the vertical scaling study three years after the 
completion of the initial scaling study. This work may occur in the optional renewal period and is 
contingent on contract renewal. 
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7.6.3.6 Scaling and Equating Studies 

As explained in Sections 7.6.1.1 and 7.6.1.2, the Florida Standards Summative Assessments will 
be pre-equated during test construction and post-equated after each spring test administration. 
The retake assessments, such as Grade 10 ELA/L or Algebra 1 Retake will be pre-equated only, 
consisting of a previously-used intact form, or another proposed design that yields a test of similar 
difficulty to the operational form. In other words, during the time frame of this ITN, there will be 
several equating activities to put the new test forms on the same scale through a series of 
intermediate equating procedures (known as equating chains). In this case, equating error from 
each of these intermediate equating procedures may accumulate to a point where the 
comparability of scale scores across time is questioned. For this reason, it is essential for the 
Department to employ practices that ensure comparable scores across time in the presence of 
equating chains. The contractor is required to submit a research plan to be conducted in spring 
2018 to investigate the extent of scale drift on the assessments. The research plan(s) must be 
approved by the Technical Advisory Committee and the Department, and the resulting report(s) 
must be submitted to the Department and presented to the Technical Advisory Committee. 

7.6.3.6.1 Investigate the Comparability between Devices Delivering Assessments 

EOC Assessments will be administered online and delivered by several different 
devices throughout the state since school districts are using many different machines 
when administering the assessments, such as laptops and desktops. The Department 
prefers to make sure these devices used in delivering the assessments are not 
providing any advantage or disadvantage to the performance of students. As a result, 
there is a need to investigate whether there is an impact of certain devices on the 
assessment results. The contractor must determine the groups of testing devices, 
based on the types of devices that work with the contractor's platform, whose impact 
will be investigated, create a plan for the study, present the plan to the Department staff 
and TAC members, and upon approval from the Department, start investigating any 
impact on item-level, test form-level, and reported score-level. The results of the study 
and contractor's recommendations must also be presented to both the Department staff 
and TAC members. 

7.6.3.6.2 Linking Florida Standards Assessments to Other Assessments Measuring the 
Same Standards 

The Department intends to compare Florida student performance on FSA to other 
students' performance on the same standards from other states, also representing the 
nation. Thus, contractor must link the scales of Florida Standards Assessments to the 
scales from other assessments measuring the same standards. 

By August of 2015, the contractor will complete a comprehensive curriculum mapping 
study to determine the degree of overlap between Florida's standards and those in 
other states in order to determine the degree of alignment. The mapping study results 
will be analyzed by the Department and contractor's staff to determine if Florida's 
standards are sufficiently like those of other states to ensure a valid assessment for 
purposes of reporting comparative information or linked data on Florida Standards 
Assessments. 

Assuming there is adequate alignment, the contractor will work with the Department to 
design, plan, and conduct a linking study. The contractor's plan for linking should 
recommend and describe the process of determining the linking methodologies, the 
data collection procedures, and how and by whom and by when all of the necessary 
steps will be completed. The linking plan must be approved by the Department and 
presented to the Technical Advisory Committee. After the linking studies are 
conducted, the contractor will prepare a technical report including a description of the 
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process, statistical background and methodology, and characteristics of the scale. The 
final report will be presented to the Technical Advisory Committee. The contractor will 
also generate data files with linked scores and prepare reports. 

If a linking study requires collecting data from either a common set of students taking 
both assessments or from different sets of students taking common set of items, the 
contractor selected through this ITN will be responsible for scoring the test forms 
administered to students for linking, including scanning/imaging, processing, 
handscoring, calibration (generation of item statistics), scoring, and production of 
student results files including linked scaled scores. The scoring will be conducted after 
the completion of all scoring and reporting for the operational tests given in Florida 
during spring of 2016. It is anticipated that the results files from the linking study will be 
provided to the Department by J uly 2016. 

As an outcome of the linking study, the contractor will provide scoring tables displaying 
how Florida's scale scores are aligned to the ones of other assessments measuring the 
same standards. Upon first use of these linking tables in reporting the results of the 
spring 2017 Florida Standards Assessments, the contractor will provide psychometric 
services to verify the reporting as well as the reliability and validity of the linked scores 
for spring 2017. 

The contractor should anticipate replicating the linking study three years after the 
completion of the initial linking study. This work may occur in the optional renewal 
period and is contingent on contract renewal. 

7.6.3.6.3 Linking Florida Standards Assessments to International Assessments 

The Department intends to compare Florida student performance on Florida Standards 
Assessments to other students' performance on International Assessments, such as 
Program for International Student Assessment (PISA) and Trends in International 
Mathematics and Science Study (TIMSS). Thus, contractor must link the scales of 
Florida Standards Assessments to the scales from international assessments. 

By August 2015, the contractor will develop a study plan for the linking Florida 
Standards Assessments to International Assessments. The contractor's plan will 
include, review of the most promising and most feasible methods likely to be used in 
international linking, discuss the technical feasibility of each method given the nature 
and design of the Florida Standards Assessments and those of international 
assessments, such as PISA and TIMSS, propose method(s) for linking the scales of 
assessments, propose alternative ways for collecting data, explain the process of 
establishing the linking relationship(s) in detail, and provide a comprehensive and 
detailed timeline for all tasks and events. The contractor will also share the report with 
the Department and itsTAC members. 

Upon approval from the Department, the contractor will finalize the plan for the linking 
and implement all of the activities. The contractor selected through this ITN will be 
responsible for scoring the test forms administered to students for linking, including 
scanning/imaging, processing, handscoring, calibration (generation of item statistics), 
scoring, and production of student results files including linked scaled scores. The 
scoring will be conducted after the completion of all scoring and reporting for the 
operational tests given in Florida during spring of 2016. It is anticipated that the results 
files from the linking study will be provided to the Department by August 2016. 

As an outcome of the linking study, the contractor will provide scoring tables displaying 
how Florida's scale scores are aligned to the ones of international assessments. Upon 
first use of these linking tables in reporting the results of the spring 2017 Florida 
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Standards Assessments, the contractor will provide psychometric services to verify the 
reporting as well as the reliability and validity of the linked scores for spring 2017. 

7.7.0 Program Management 

Respondents should provide their overall description of the approach to manage, implement, and support 
Florida's assessment program that specifically addresses the entire scope of work in this ITN. Respondents 
should identify and describe all resources available to support all program activities. A management plan for 
the contract should be incorporated into the respondent's reply. In the event that multiple contracts are 
awarded, additional program management capacity will be required to coordinate among the other contractors 
and the Department. 

Respondents should designate a Senior Program Manager who will have overall, daily responsibility for all 
processes and deliverables under the full scope of this program. 

The Chief of the Department's Bureau of K-12 Assessment will be assigned as the program manager of the 
contract resulting from this ITN under the overall supervision of the Director of Assessment, Division of 
Accountability, Research and Measurement. 

7.7.1 Provisions Governing the Work of the Contractor 

The procedural requirements identified in this section govern the work of the contractor. This section 
establishes specific requirements and general principles by which the program will be managed. 

The work described in Sections 7.2.0 through 7.6.0 reflects design, development, validation, and 
implementation requirements of Florida's standards-based testing program. In preparing replies, respondents 
should consider the following provisions governing the activities required by these sections. 

a. All of the processes used by the contractor to complete the work identified in Sections 7.2.0 through 7.7.0 
must be approved by the Department. All products must be approved by the Department at milestone 
stages of development as described in the Production Schedule developed at the beginning of the contract 
The Department reserves the right to specify details of tasks and products. 

b. Design control of all aspects of this program remains the responsibility of the Department Changes in the 
design made after the award of the contract will be at the discretion of the Department. 

c. The Department reserves the right to require changes in materials, produced in any medium, at any stage 
of the program prior to final printing, production, or deployment. 

d. Products and processes are modified to some extent for each assessment. For example, modifications 
may be made to the design, format, or wording of printed and online products from one administration to the 
next. All such modifications are to be expected as part of the scope of this program and will be at 
contractor expense. Modifications may also be required for other products and procedures, such as data 
analyses, file formats, and report designs. Consequently, all printed products developed under this contract 
are to be only produced in quantities for the particular assessment, unless otherwise specified in writing by 
the Department 

7.7.2 Contractor Staff and Responsibilities 

The work described in the ITN constitutes a large, complex, challenging program that requires year-round 
activity and close attention to overlapping tasks for different years and the impact on project resources. The 
contractor is expected to provide a sufficient number of highly-qualified personnel to work closely with the 
Department to manage the contract. Respondents should carefully consider whether they have sufficient and 
knowledgeable human resources to complete the work activities of the ITN and should demonstrate in their 
replies that they have sufficient and knowledgeable human resources to complete program work. Companies 
should consider bidding only if they are committed to assigning the number and quality of staff and staff time 
required to complete a program of this magnitude. 

The contractor must assign professional staff members who are qualified, experienced, and capable of 
providing technical assistance to the Department. Please note the following staff requirements: 
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a. The contractor will assign at least one (1) full-time senior program manager, four (4) full-time 
project managers, and two (2) full-time administrative assistants. The full-time senior program 
manager will concentrate on ensuring quality and timeliness of the full range of services and products 
required by this ITN. The contractor's management team is expected to lead project and support staff 
and to manage implementation activities throughout the contract period. Although this section outlines 
the minimum staffing requirements for this program, the contractor must provide adequate staff to 
maintain the highest quality products and must deliver these products according to the Department's 
timelines. 

b. It is anticipated that the four (4) full-time project managers will focus on (a) test development, (b) test 
administration, (c) scoring and reporting, and (d) computer-based testing. Respondents may propose a 
modification to this project assignment for the managers. 

c. Implementing the program requires considerable administrative activities involving the maintenance of 
schedules, the development of specifications, production of management reports and documents, 
arrangement of meeting and travel for educators and Department staff, and communication with district 
and other personnel. Administrative assistants are necessary to serve in a dedicated role as meeting 
organizers and as direct contacts for the many Florida citizens that may attend assessment meetings 
throughout the year. The Department requires a single point of contact to discuss each meeting's 
requirements and venue options and to communicate with participants throughout the travel process, and 
to address issues that may require late-notice changes in travel plans. 

d. The managers should be members of the regular organizational staff with experience within the company 
or institution in coordinating the types of activities identified by this ITN. In their replies, respondents are 
expected to demonstrate that the individuals identified as managers have sufficient authority across 
departments within the organization to ensure that the work of the contract has the necessary priority to 
be completed with the highest quality and on time. 

e. Respondents should include pertinent staffing charts in their replies that show the regular institutional 
organization staffing and the proposed organization of program staff for this work. In these charts, the 
full-time equivalency devoted to work under the contract for each staff member with significant 
responsibilities will be identified. Persons proposed to fill all key positions will be identified by name. 

f. Respondents should indicate, by name, the professional personnel to be responsible for major contract 
activities, with an estimation of the amount of time, as a percent of full-time, each person will devote to 
each task. This information will be included in a separate chart in the management reply. An appendix to 
the technical reply should contain one (1) page vitae for each of the professional personnel, including 
subcontractors, to be assigned to the program, indicating relevant educational background and 
professional experience. 

g. In a separate chart(s), respondents should identify the full extent of staff resources to be allocated to the 
program to conduct activities related to the item and test development; computer-based test system and 
content support; ancillary materials development; scanning, imaging, data processing, and reporting 
activities; and scoring, equating, and technical analysis/special studies activities for each year of the 
program. The chart should identify the major activities by month and the number of staff by category 
and the amount of time in full-time equivalencies (equivalent person-days) assigned to complete the 
activities. 

h. In addition, the contractor must provide for five (5) contractor staff positions funded through the 
reimbursable category and described in Section 7.7.9.6. 
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7.7.3 Subcontractors 

In addition to any subcontractors required in this ITN, the prime contractor may choose to employ one (1) or 
more additional subcontractors for the completion of these tasks. If the respondent proposes to employ a 
subcontractor(s), the qualifications and experience of the subcontractor(s) should be documented in the reply 
at the same level of detail as those of the respondent. 

The prime contractor will assume responsibility for all services offered in the reply whether or not they are 
performed or produced by the contractor or by subcontractors. The Department will consider the prime 
contractor to be the sole point of contact for contractual matters, including payment of any and all charges 
resulting from the contract. 

The respondent should describe how the work of all subcontractors will be monitored and guided to ensure 
adherence to requirements in this ITN, including resulting specifications and schedules. The contractor will 
provide for quality assurance activities in oversight of the subcontractor's performance. 

A separate chart in the reply should identify all of the subcontractors proposed to be involved in the program, 
the assessment components they will be involved with, and the specific services they will provide. 
Throughout the life of the contract, any proposed changes in subcontractors must be approved by the 
Department. The contractor will provide the Department with a qualifications and experience summary for 
review and consideration for acceptance in advance of a proposed subcontractor's start of work on program 
processes or deliverables. As part of the Annual Report, all subcontractors, their services, and Departmental 
approval status will be listed. 

A detailed description of the subcontracted services to be provided under the ensuing contract should be 
included in the reply. All subcontracted services should be documented. Each service (e.g., printing, computer 
operations, test development, psychometric services, legal counsel, quality control, reporting) should be fully 
described, including resources, level of service, scheduling, and quality commitments. The respondent should 
identify the proportion of materials, etc., to be generated by any and all involved subcontractors. 

No contractor- or subcontractor-provided work related to customer service or to any secure document or data 
will be provided by a persons or agencies physically located outside of the United States. 

7.7.4 Program Management Communication 

Weekly program management conference calls will be conducted at an agreed upon day and time in order to 
ensure that all requirements and timelines are maintained and to address needs for problem solving and time- 
sensitive adjustments. The Department may determine periodically that weekly, focused conference calls are 
required as activities dictate. The contractor will provide a toll-free conference line for all calls between contractor 
and Department staff required for the effective coordination of the assessment program. 

To meet timelines for completing tasks, both the Department and the contractor will need to use secure 
electronic communications, including, but not limited to, a secure FTP site, a website, a teleconferencing site, 
etc., provided by the contractor in addition to overnight express delivery of materials as necessary. In 
addition, the Department and the contractor will need access to a secure site that can be used with districts 
and with subcontractors to transfer secure materials. 

The contractor and the Department may determine that, for selected meetings, video teleconference meetings 
will be more time and cost efficient than travel to a common meeting site. In this case, the contractor will 
coordinate all logistics and provisions for the teleconference. 

To assist Department staff with communication during on-site visits to the contractor's offices or other work 
sites, the contractor will provide access to lockable office space, including a telephone with a speaker, a fax 
machine, and a computer with Internet access and a local printer. 
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7.7.5 Program Management Meetings 

The successful operation of the program will require regular management meetings between the contractor and 
Department staff. At the Department's request, the management meetings will take place in Tallahassee, the 
contractor's headquarters or other worksite, or in a location convenient and agreed upon by the contractor and the 
Department. Contractors should anticipate that the first comprehensive program management meeting will take 
place at the contractor's headquarters within thirty (30) days of contract execution. 

The contractor must expect that up to fifteen (15) of their staff will travel to attend approximately two (2) 
comprehensive program management meetings per year. The contractor will also send all relevant staff to up to 
four (4) additional annual management meetings that are project specific, such as the scoring and reporting 
debrief. All relevant subcontractors must also attend these meetings. The contractor will be responsible for the 
logistics, facilities, and travel costs of their staff and required subcontractors' staff for all management meetings. 
These meetings will also involve relevant Department staff. Provisions will be made for electronic participation of 
any staff member unable to travel to these meetings. In the event that multiple contracts are awarded, the 
contractor should expect part of the meetings will include all other contractors and Department staff, and that part 
of the meetings will be reserved for one-on-one meetings between the Department and each contractor to focus 
on topics and issues specific to each contractor. 

The cost reply should include funding specifically for Department staff travel for program management meetings 
and other oversight activities. Program oversight activities include work related to overall program management 
management of key processes, including development, test administration, scanning, handscoring, scoring, 
programming, printing, and reporting. The contractor will be responsible for paying the cost of travel, lodging, and 
per diem for up to ten (10) Department staff members to attend up to fifteen (15) trips annually. The cost of the 
trips has been based on travel between Tallahassee and the contractor's headquarters or between Tallahassee 
and the contractor's or subcontractor's headquarters, scanning/processing sites, or scoring/handscoring sites. 
Funds set aside for this purpose are described in Section 7.7.9.5. 

7.7.6 Program Management Reports 

Five types of reports are to be prepared by the contractor to facilitate and document program management 
communications with the Department: (1) the Program Schedule, (2) the Weekly Management and Weekly Action 
Item Reports, (3) a Technical Report, (4) Annual Program Review, including the Operational Addendum, and (5) 
the Accounting Report. In addition to written reports, the Department may require the contractor's participation 
in presentations to Department senior management or other entities at places and times determined by the 
Department. 

1. The Program Schedule will be developed for each year of the program and will include both high-level 
management summary information as well as specific dates associated with critical processes, milestones, 
and deliverables for individual projects listed in the program management plan. The Program Schedule will 
be derived from the program management plan at the beginning of each test administration year, and will be 
updated quarterly. The Department expects submission of the Program Schedule in Microsoft Excel. If 
significant changes occur between quarterly submissions of the program schedule, the Department may 
request an updated schedule reflecting the changes at any time. 

2. The contractor will provide a Weekly Management and Action Item Report to the Department. A template for 
this report will be provided to the contractor, and the Department reserves the right to modify the design, 
format and content of this report at any time. The Weekly Management Report and Action Item Report will 
report the status of recent and upcoming tasks, highlighting those leading up to deliverables and critical tasks, 
report progress on program issues, identify unresolved issues, and maintain a list of requests requiring 
change orders or amendments. The weekly report will be provided to the Department by no later than noon 
one day prior to the weekly program management conference calls and will reflect the previous week's work. 

3. The contractor will produce a Technical Report and provide the initial draft document to the Department by 
September 30 of each year of the contract arising from this ITN. The Department will work with the 
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contractor to identify the specific information to be provided in the Technical Report each year. A 
proposed report structure is provided in Section 7.6.3.2. 

4. The contractor will produce two copies of the Annual Program Review each year to be provided only on 
CDs/DVDs/jump drives inside plastic sleeves in loose-leaf binders. Along with other materials described 
below, the Program Review will include a record of the contractor's services and deliverables in response to 
contractual/ITN requirements. This review must identify major program accomplishments and successes, as 
well as areas of the program that require changes and improvement in the upcoming year. The review is 
intended to be a history of program events and to describe recommended improvements or changes for 
subsequent years. The annual review will be based on information gathered by the contractor's program and 
project management staff throughout the year. The review will address all aspects of program service and 
products including management technology, interpretive products, communications, development test 
production, test administration, materials shipping/pickup, scoring, and reporting processes. It will also follow 
the annual administrative cycle, beginning with fall and ending with the following summer. 

The Annual Program Review will include: 

- details of meetings held with educators (dates, locations, purpose) 

- milestones attained/deliverables completed for each project area 

- problems encountered, root cause analyses, and corrective action(s) taken 

- performance metrics reports as required by the Department noting changes over time 

The Annual Program Review will include electronic files of deliverables and documentation on services 
required by this ITN, beginning with the fall test administration and ending with the summer test 
administration. It will include: 

• PDFs of the testand answer books, 

• PDFs of all screenshots for all computer-based test approved forms, 

• PDFs of all passage booklets, reference sheets, large print test books, one-item-per-page test books, 
large print reference sheets, large print response grid instruction sheets, and CBT work folders, 

• PDFs (untagged and tagged, if applicable) of all interpretive and ancillary products, 

• Native application files of all interpretive and ancillary products, 

• PDFs, or other appropriate file formats such as video, of all other products, including memoranda and 
report templates, 

• Missing materials and resolution reports, 

• Final Program Schedule reflecting planned vs. actual dates of task completion, 

• Weekly Management and Action Item reports, 

• File Layouts, and 

• Specifications documents. 

5. The collected Accounting Reports will provide a summary of the Quarterly Accounting Reports, including 
expenditures by program task and level of effort as well as expenditures and balances of Reimbursable 
Funding Categories identified in Section 7.7.9, including balances resulting from executed change orders and 
contract amendments. 

7.7.7 Meetings Involving Florida Educators and Other Citizens 

The contractor will be responsible for arranging, organizing, and paying the cost of bringing participants to 
specific meetings as identified in this ITN. This section identifies points at which meetings with groups of 
Florida educators and other citizens will be conducted to propose development and/or management 
guidelines, to draft products, or to review work and products produced by the contractor. The Department will 
designate Florida educators, and in some instances Florida citizens or consultants from outside the state, to 
participate in these meetings. 
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All meetings are facilitated by Department staff unless determined otherwise by the Department. Contractor 
staff will attend the meetings with Florida educators, as required. The contractor is responsible for on-site meeting 
organization, support, and management. Generally, the contractor's manager or team leaders for the program 
area and an administrative assistant responsible for meeting planning will be present at meetings for overall 
management and support purposes. The contractor will also be expected to send staff appropriate to assist and 
participate in each meeting. Generally, for all meetings involving separate subject/grade groups, the contractor will 
provide the appropriate professional staff to take part in each of the separate subject/grade groups. The contractor 
will be responsible for all expenses associated with contractor staff travel and participation. All expenses must be 
in accordance with the Department of Financial Services Guidelines for State Expenditures which prohibits the 
purchase of food and beverages and other unallowable costs. The Guideline may be accessed through the 
following link: http://www.myfloridacfo.com/aadir/reference guide/ . 

The contractor will be responsible for arranging, organizing, attending, and paying travel, lodging and per diem 
costs required to bring these participants together for the meetings. For meetings to be held in Florida, they 
will typically be in a central Florida location, a hub city (e.g., J acksonville), or in Tallahassee. 

Table 7.7.1 provides information about the number, expected length, approximate number of participants, and 
estimated cost of such meetings. 



Table 7.7.7 Meetings of Educators and Other Citizens 



Table 7.7.7 Meetings of Educators and Other Citizens 




Meeting 


Area 


ITN 
Ref. 


Mtgs/ 
Year 


Base 
Contract 


Renewal 


Days/ 
Mtg 


Approx # 
of 

attendees/ 
ind mtg 


Likely Site 


l 


Item/ Passage 
Bias 


English 
Language 
Arts/ Literacy 

(ELA/L), 
Mathematics 


7.3.4.1 


1 


Oct 2015, 
2016, 2017 


Oct 
2018, 
2019, 
2020 


2 


30 


J acksonville 
or Central 
Florida 


2 


Item/ Passage 
Community 
Sensitivity 


ELA/L, 
Mathematics 


7.3.4.1 


1 


Apr 2015, 
2016; May 
2017 


May 
2018, 
2019, 
2020 


2 


30 


J acksonville 
or Central 
Florida 


3 


Item Content 
Review 


ELA/L, 
Mathematics 
Test Items 


7.3.4.2 


1/ 

subjects 
grade 


Oct 2015, 
2016,2017 


Oct 
2018, 
2019, 
2020 


5 


8 


J acksonville 
or Central 
Florida 


4 


Passage 
Content 
Review 


ELA/L 


7.3.4.2 


1 


Aug 2015, 
2016, 2017 


Aug 
2018, 
2019, 
2020 


5 


15 


J acksonville 
or Central 
Florida 


5 


Data Review 


Mathematics 


7.3.4.3 


1 


May orj un 

2015, 
2016, 2017 


May or 
J un 
2018, 
2019 


2 


9 


Tallahassee 


6 


Standard 
Setting 


ELA/L and 
Mathematics 


7.6.3.4 


1/ 

subject 


Aug 2015 


N/A 


5 


300 


Central FL 


7 


Standards 
Reactor 


ELA/L and 
Mathematics 


7.6.3.4 


1 


Aug 2015 


N/A 


3 


30 


Tallahassee 
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Table 7.7.7 Meetings of Educators and Other Citizens 




Meeting 


Area 


ITN 
Ref. 


Mtgs/ 
Year 


Base 
Contract 


Renewal 


Days/ 
Mtg 


Approx # 
of 

attendees/ 


Likely Site 


8 


Content 
Advisory 


ELA/L, 
Mathematics 


7.3.4.4 


3 per 
subject/fi 
rstyear; 

2 per 
subject 
after first 

year 


TBD 


TBD 


3 


25/subject/ 
meeting 


Tallahassee, 
J acksonville, 
or Central FL 


9 


Technical 
Advisory 
Committee 


Technical 


7.6.0 


2 


Spring and 
Nov 2015, 
2016, 2017 


Spring 
and Nov 
2018, 
2019 


2 


20 


Tallahassee, 
J acksonville, 
Central FL, or 
site of F ERA 


10 


Field Test 
Rangefinding 


ELA/L, 
Mathematics 


7.5.4.5 


21 
subject 


May and 
Sep 2015, 
2016, May 
2017 


Sep 
2017; 
May and 

Sep 
2018, 
2019, 

May 
2020 


5 


10/subject 
per grade 
(3 

grades/cou 
rses per 
subject) 


Tallahassee 


11 


Operational 
Rangefinder 
Selection 


ELA/L, 
Mathematics 


7.5.4.5 


1/ 

subject 


Sep 2015, 
2016, 


Sep 
2017, 
2018, 
2020 


5 


8 


Tallahassee 



7.7.8 Communication between the Contractor and Districts 

Florida has 67 school districts conforming to the boundaries of Florida's 67 counties. Within each school 
district is a district coordinator of assessment assigned to work directly with the Department to implement 
statewide assessment programs. All aspects of test administration and reporting are administered within the 
districts through the district coordinator of assessment. Other district personnel involved in statewide test 
administration include one testing coordinator per school and the teachers or guidance counselors who serve 
as test administrators. While there are some testing sites that are not part of the public school system (e.g., 
private schools), most communication will still be directed through the district coordinator of assessment. 

All communication between the contractor and school districts must be directed through or receive prior approval 
by the Department. Memos required to implement various aspects of the assessment will be prepared by the 
contractor and approved by the Department prior to release. The contractor will be required to email, mail, or fax 
memoranda or sets of instructions to district coordinators of assessment as instructed by the Department. 

Districts will be instructed to communicate directly with the contractor only for the purpose of clarifying meeting 
travel logistics, ordering additional testing materials, correcting computer-based system problems, or to report 
problems with the delivery/pickup of materials. Forthis purpose, the contractor will dedicate for the use of Florida's 
district and school personnel a toll-free help desk telephone, operated Monday through Friday 6:00 am through 
8:00 pm, Eastern Time, a toll-free fax number, and a web-based help site available during the same hours cited for 
the help-desk telephone. This customer service is distinct from that described in Section 7.4.10. 
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7.7.9 Reimbursable Funding Categories 

Six funding categories are designated for specified program functions and may be used only for those 
functions. Use of these funds requires authorization by the Department contract or program manager or 
program area leads. Authorization may be provided by specific email or other written communication, or by 
Department approval of a specific plan for a meeting(s) or activities requiring use of these funds. Amounts are 
designated for administration tasks. Respondents should include these amounts in their cost replies. 

Monies not expended from these funds will be deducted from final contract payment at the conclusion of the 
contract during contract resolution. The reimbursable fund balances will be reported by the contractor 
quarterly with each invoice submission in the Accounting Report. The amounts specified in this section may 
be increased or decreased by an appropriately negotiated amendment at any time during the lifetime of the 
contract. The Accounting Report will indicate expenditures from, and the balances remaining, in these funds. 
The final contract invoice will be accompanied by the final balance sheet for these funds. The amounts 
allocated for these funds should be identified separately in the respondents' cost reply. 

7.7.9.1 Consultant Services 

The contractor will provide to the Department technical advice and consultant service, as required, 
from outside the contractor's staff. The contractor will recommend and the Department will select 
the consultant(s) to be used, who may or may not have been recommended by the contractor. A 
contract between the consultant and the contractor will be required and will include the amount 
allocated for honorarium, travel, lodging, and per diem as authorized by the Department. 
Reimbursement for travel will be at rates authorized by Section 112.061, F. S., for travel and per 
diem. The respondent should include in the reply the amount of $84,375.00 as a fixed amount for 
the base contract period and $28,125.00 as a fixed amount for each year of the optional renewal 
period. 

7.7.9.2 Contingency Services 

It is possible that unforeseen circumstances will make it either necessary or desirable to perform 
tasks not specifically anticipated at this time, for example, data analyses beyond those described 
herein, unexpected data processing, acquisition of additional materials, or programming for test 
functionality not yet envisioned. These circumstances cannot be defined in advance, and the 
Department must be able to request such extra services in a timely fashion so as not to impede 
implementation of the assessments. In the reply, the respondent should include $84,375.00 as a 
fixed amount for the full term of the base contract period and will include $28,125.00 as a fixed 
amount for each year of the optional renewal period. 

7.7.9.3 Author's Alterations and Design Control 

The Department reserves the right to make changes in materials at any stage of the program prior 
to final production or delivery. Any changes to the Department approved, production-ready product 
(e.g., digital proofs or final computer-based test files) that are required due to contractor error will 
be paid for by the contractor. Otherwise, changes made to the Department approved, production- 
ready product will be paid for as author's alterations. If any alterations are necessary, the 
Department will identify in writing to the contractor the alterations to be made. In the reply, the 
respondent should include $84,375.00 as a fixed amount for the full term of the base contract 
period and will include $28,125.00 as a fixed amount for each year of the optional renewal period. 

7.7.9.4 Provision for Meetings with Florida Educators 

The respondent will include a fixed amount to cover the costs of travel and per diem for Florida 
educators, Florida citizens, and others participating in the meetings identified in Section 7.7.7. The 
contractor will be responsible for the travel reimbursement for participants at rates authorized by 
Section 112.061, F. S., for travel and per diem. In the reply, the respondent should include 
$5,491,496.00 as a fixed amount for these meetings with Florida educators during the base contract 
period and $1,830,498.00 for each year of the optional renewal period. 
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7.7.9.5 Provision for Meetings with Department Staff 

A fixed amount will be included in the reply to cover the costs of travel and per diem for the 
Department of Education staff or others participating in meetings with the contractor on behalf of 
the Department. Reimbursement for staff will be at rates authorized by Section 112.061, F.S. 
These funds will not be used to fund travel and per diem of the contractor's staff. These funds will 
be expended only with the approval of the Department. In the reply, the respondent should include 
$438,750.00 as a fixed amount for ten (10) staff members to attend as many as fifteen (15) 
meetings per year requiring four (4) travel days each during the contract during the base contract 
period and $146,250.00 for each year of the optional renewal period. 

7.7.9.6 Contractor Staff Positions 

To improve work flow, work quality, and communications, the contractor will fund salary and 
benefits for five (5) professional positions to work under the direction of the Bureau of K-12 
Assessment within the Florida Department of Education. These positions will be three editors and 
two Systems Project Consultants. The Department will work closely with the contractor to interview 
and select individuals who are acceptable to both parties. The contractor will employ its standard 
recruiting, screening, and hiring procedures to secure individuals to fill these positions. The 
employees will be full-time employees of the contractor. The employees will receive benefits 
typically provided to the contractor's employee classification system. The employees will be paid 
at a rate comparable to similarly skilled and experienced Department employees using the 
contractor's employee classification system. 

The basic duties of the Computer-Based Test Editor (1) and the Materials Editors (2) include: (a) 
write and edit materials for content, such as items, tests, reports, training materials, brochures, 
scripts, or on-screen presentations; (b) develop, analyze, and select appropriate information and 
present it in a form and at a level suitable for the intended audience; (c) review written products 
and materials for conformity with standards of objectivity, style, and presentation; and, (d) perform 
a variety of editorial duties related to editorial production. Candidates must possess one (1) year of 
specialized experience directly related to these duties. Candidates may substitute two (2) full 
academic years of related graduate level education in lieu of the work experience requirement. 

The basic duties of the Systems Project Consultants (2) include: (a) participating in weekly 
meetings and communicating issues between the FDOE and the contractor's program managers, 
(b) making suggestions for improvement to the contractor's procedures to aid FDOE staff in data 
verification activities, (c) assisting FDOE staff debugging SAS computer programs and analyzing 
data files, (d) traveling, at the contractor's expense, with FDOE staff to check scanning at the 
contractor's site, (e) assisting in the verification of reports of results using SAS computer programs, 
(f) assisting in the verification of posting of files and reports on the portal, and (g) communicating 
issues with District Coordinators of Assessment related to scoring and reporting activities and 
issues. Candidates must have a thorough understanding of large scale standardized testing 
programs and data verification techniques. The candidate must also have experience managing 
data activities such as those required for this project, verifying data files, and using SAS, Excel, 
and other Microsoft Office products. 

The respondent should include in the response $1,800,000.00 as a total amount during the base 
contract period and $600,000.00 for each year of the optional renewal period. 

7.7.10 Final Delivery of Materials 

The contractor agrees to deliver to the Department, upon request, all materials and products in all 
forms that are developed for and used in conjunction with this program, including test books, 
answer documents (paper and PDF), final electronic source files of interpretive products, software 
source code, and ancillary materials, PDFs, and electronic files within 30 days following 
acceptance by the Department of the final report for the program. Payment of the final program 



Florida Standards Assessments, ITN 2014-47 



95 



invoice will not be made until all materials and certification of destruction, as appropriate, are 
received and approved by the Department and final payment resolution is agreed to by both 
parties. 

At the completion of the contract, all data files, including those containing items or graphics, 
remaining in the possession of the contractor will be destroyed once all handoffs are made to the 
next contractor. Written verification of destruction will be provided to the Department as part of the 
final contract report. Destruction will not take place until authorized by the Department. 

7.8 DELIVERABLES (See Attached Section 7.8 Deliverables provided in an Excel Spreadsheet.) 



SECTION 8 - OPENING, EVALUATION AND AWARD 

8.0 REPLY OPENING 

TECHNICAL & PRICE REPLIES WILL BE OPENED BY THE DEPARTMENT'S BUREAU OF CONTRACTS, 
GRANTS AND PROCUREMENT MANAGEMENT SERVICES PERSONNEL AT 325 WEST GAINES 
STREET, 332 TURLINGTON BUILDING , TALLAHASSEE, FLORIDA, AS SPECIFIED IN SECTION 3.4 
CRITICAL EVENT DATES 

8.1 REPLY EVALUATION AND NEGOTIATION PROCESS 

Using the evaluation criteria specified below, in accordance with Section 287.057, F.S., the Department shall 
evaluate and rank responsive Replies and, at the Department's sole discretion, proceed to negotiate with one 
or more Respondent(s) selected, as follows: 

A. Selected Respondent(s) will be invited to commence contract negotiations. If necessary, the Department 
will request revisions to the approach submitted by the selected Respondent(s) until it is satisfied that the 
contract will serve the Department's needs. The process will continue until a contract is negotiated and 
executed. The Department may in its sole discretion, award and enter into contracts with more than one 
Contractor, if in the best interest of the state. 

B. The Department reserves the right to negotiate with all responsive and responsible Respondents, serially 
or concurrently, to determine the best-suited solution. The ranking of Replies indicates the perceived 
overall benefits of the proposed solution, but the Department retains the discretion to negotiate with other 
qualified Respondents as deemed appropriate. 

C. Before award, the Department reserves the right to seek clarifications, to request Reply revisions, and to 
request any information deemed necessary for proper evaluation of Replies. Respondents may be 
requested to make a presentation, provide additional references, provide the opportunity for site visits, 
etc. The Department reserves the right to require attendance by particular representatives of the 
Respondent. Any written summary of presentations or demonstrations shall include a list of attendees, a 
copy of the agenda, and copies of any visuals or handouts, and shall become part of the Respondent's 
Reply. Failure to provide requested information may result in rejection of the Reply. 

D. The focus of the negotiations will be on achieving the solution that provides the best value to the state. 

E. In submitting a Reply Respondent agrees to be bound to the terms of this ITN, however, the Department 
reserves the right to negotiate different terms and related price adjustments if the Department determines 
that it is in the state's best interest to do so. 

F. The Department reserves the right to reject any and all Replies, if the Department determines such action 
is in the best interest of the state or the Department. The Department reserves the right to negotiate 
concurrently or separately with competing Respondents. The Department reserves the right to accept 
portions of a competing Respondent's Reply and merge such portions into one project, including 
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contracting with the entities offering such portions. The Department reserves the right to waive minor 
irregularities in Replies. 

Criteria for Evaluation 

The Department shall evaluate and rank responsive Technical Replies and score them on a scale of 1 to 100 
using the following criteria. The Department anticipates awarding the Contract (if any award(s) is made) after 
negotiations to the responsible and responsive vendor determined to provide the best value to the state. 

Each Respondent will receive one score for all criteria, other than price, regardless of the number of 
assessment components it proposes to deliver. This score will constitute the Total Technical Score for the 
Respondent. Respondents will also receive a separate price evaluation for each assessment component it 
submits a price for (A-E in Attachment 1). The Department will add the Total Technical Score for each 
Respondent to the price evaluation score for an assessment component to determine the total score for the 
Respondent for that assessment component. Respondents will be ranked for each assessment component in 
this manner. 

The following criteria will be used to evaluate and rank Replies. 



A) Executive Summary N/A 

B) Qualifications and Experience 10 points 

C) Technical Plan 50 points 

D) Management Plan 15 points 

Total 75 points 

E) Price 25 points 

Total 25 points 



Price analysis is conducted through the comparison of Price Replies submitted. The Department shall score 
responsive Price Replies up to 25 points using the below. 

Price analysis is conducted through the comparison of Price Replies submitted. 

The method of awarding points for the Price Reply will be done in accordance with the provisions of Section 
287.0572, F.S., which requires use of the present value methodology. The present value discount rate will be 
used in the computations and evaluation. To determine that discount rate, use the rates identified in Release 
H.15, Select Interest Rates (Weekly), available online at http://www.federalreserve.gov/releases/hl5/ . Use the 
most recent copy of the Federal Reserve Bulletin published at the time of issuance of the ITN. See the 
"Critical Event Dates" for the release date. 

The maximum points will be awarded to the lowest acceptable Price Reply. Replies with higher costs will 
receive the fraction of the maximum points proportional to the ratio of the lowest Price Reply to the higher 
Price Reply. The fractional value of points to be assigned will be rounded to one decimal place. The criteria 
for price evaluation shall be based upon the following formula: 

(Low Price/Respondents Price) x Price Points = Respondent's Awarded Points 

The total maximum number of points that can be earned in the evaluation process is 100 points. 

8.2 EVALUATIONS 

The ITN is designed to assess the most points to the Respondent presenting the best solution for the required 
services. The Evaluation Committee will consider only those Replies provided after the preliminary 
administrative review (See SECTION 6.2.1) first completed by the Department's Bureau of Contracts, Grants 
and Procurement Management Services. 

Each member of the Evaluation Committee will be provided a copy of each Technical Reply. Replies will be 
evaluated on the criteria established in the section above entitled "Criteria for Evaluation" in order to assure 
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that Replies are uniformly rated. The Evaluation Committee will assign points, utilizing the technical 
evaluation criteria identified herein and the Procurement Office will complete a technical summary. 

The Department reserves the right to request oral presentations/seek clarification on any Reply submitted. 
Information requested and received will be evaluated by the Committee based on the criteria established in 
SECTION 6.2.2 above. During this stage Respondents will be asked to provide any clarifications needed by 
the Evaluation Committee to assist in evaluating their Reply. Information received in this stage will be added 
to the Respondent's Reply and evaluated as a part of the appropriate section above. 

8.3 POSTING OF INTENDED AWARD 

The intended award will be posted in the Bureau of Contracts, Grants and Procurement Management 
Services, 325 West Gaines Street, 332 Turlington Building, Tallahassee, Florida and on the VBS and will 
remain posted for a period of seventy-two (72) hours. (See SECTION 3.4 CRITICAL EVENT DATES .) 

8.3.1 Protest of Intended Award 

Any Respondent who is adversely affected by the Department's recommended award or intended decision 
must file a written "Intent to Protest" with the Department at the address of posting. See SECTION 4.4 
PROCUREMENT PROTESTS/NOTICE OF RIGHTS for protest information. 

8.3.2 Inability to Post 

If the Department is unable to post as defined above, the Department will post a public notice on the VBS. 
The Department will provide written notification of any future posting in a timely manner. 

8.4 AWARD OF THE CONTRACT 

Services will be authorized to begin when the Contractor receives a fully executed contract from the 
Department's Contract Administrator. 

* * * * 
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APPENDIX A: 

TRANSITION TO COMPUTER-BASED TESTING AND ADDITIONAL TEST ADMINISTRATION DATES 



All high school and End-of-Course assessments are currently computer-based assessments and are anticipated to 
remain as such. Grade 11 English Language Arts/Literacy and Algebra 2 End-of-Course Assessments will be 
implemented as computer-based assessments. 



Table A.l: Transition to FSA Computer-Based Tests in Florida 
Grades 3-8: ELA/L and Mathematics 





Cuhiprt 


2013-2014 


2014-2015 

bVlt 4U1J 


2015-2016 


2016-2017 

favlU favl/ 


2017-2018 


3 


ELA/L 


PBT 


PBT 


PBT 


PBT 


Transition to 
CBT 


Mathematics 


PBT 


PBT 


PBT 


Transition to 

1 1 Ul IJIUvl 1 Lv 

CBT 


CBT 


4 


ELA/L 


PBT 


PBT 


Transition to 
CBT 


CBT 


CBT 


Mathematics 


PBT 


PBT 


PBT 


Transition to 
CBT 


CBT 


5 


ELA/L 


PBT 


Transition to 
CBT 


CBT 


CBT 


CBT 


Mathematics 


CBT 


CBT 


CBT 


CBT 


CBT 


6 


ELA/L 


CBT 


CBT 


CBT 


CBT 


CBT 


Mathematics 


Transition to 
CBT 


CBT 


CBT 


CBT 


CBT 


7 


ELA/L 


CBT 


CBT 


CBT 


CBT 


CBT 


Mathematics 


PBT 


Transition to 
CBT 


CBT 


CBT 


CBT 


8 


ELA/L 


Transition to 
CBT 


CBT 


CBT 


CBT 


CBT 


Mathematics 


PBT 


PBT 


Transition to 
CBT 


CBT 


CBT 



Key 

CBT =Was computer-based test in prioryear 

Transition to CBT = First year of computer-based test in this grade/ content 

PBT = Administered as a paper-based test 
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The following dates are for reference only and refer to assessments that are not included as part of this ITN. 

Dates for the assessments that are included in this ITN are provided in Table 7.2.2. 



Table A.2: Additional Administration Dates for FCAT 2.0 and Florida EOC Assessments 
(Assessments not included in this ITN) 




FCAT 2.0 


Florida End-of-Course Assessments (U.S. History, 
Civics, Biology 1, Algebra 1 Retake, Geometry Retake) 


Year 


Fall 
Reading 
Retake 


Spring 
Reading 
Retake 


Spring Science 


Fall 


Winter 


Spring 


Summer 


zUlD-lo 


Oct. 5-16, 
2015 


April 11- 
lS, 2016 


April 11-15, 
2016 


Sept. 14-25, 
2015 


Nov. 30- 
Dec. 18, 
2015 


April 25-May 
27, 2016 


July 11-22, 
2016 


2016-17 


M A 

NA 


M A 

NA 


April 17-21, 
2017 


Sept. 12-23, 
2016 


Nov. 28- 
Dec. lb, 
2016 


April 24-May 
26, 201/ 


July 10-21, 
201/ 


2017-18 


NA 


NA 


April 16-20, 
2018 


Sept. 11-22, 
2017 


Nov. 27- 
Dec. 15, 
2017 


April 30-May 
25,2018 


July 16-27, 
2018 


2018-19 


NA 


NA 


April 15-19, 
2019 


Sept. 10-21, 
2018 


Nov. 28- 
Dec. 19, 
2018 


April 29-May 
24,2019 


July 15-26, 
2019 


2019-20 1 


NA 


NA 


April 13-17, 
2020 


Sept. 9-20, 
2019 


Dec. 2-20, 
2019 


April 27-May 
22,2020 


July 13-24, 
2020 


2020-21 1 


NA 


NA 


April 12-16, 
2021 


Sept. 14-25, 
2020 


Nov. 30- 
Dec. 18, 
2020 


April 26-May 
21,2021 


July 12-23, 
2021 


1 Optional Renewal Period 
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APPENDIX B 

CRITICAL AND EXTREMELY CRITICAL ACTIVITIES 



Final dates for these activities and a final activities list will be determined in the negotiation process of this ITN and included in the final contract. The table below 
describes the critical and extra critical tasks, the frequency for distribution, and any notes about the approximate delivery dates. Failure to provide these materials 
by the agreed-upon dates will be cause for the assessment of liquidated damages. 



Critical (C)or 
Extra Critical 




Delivery Notes (Approximations) 



Delivery of practice tests and sample test 
materials to districts. 


C 


Varies by deliverable 


By November/December for assessments that are 
introduced in the spring, regular sample tests (P DF or 
computer-based) are delivered the initial year of the 
assessment or the first year of a format change (i.e., 
transition from paper-based to computer-based). 

Special format print materials (e.g., Braille, large print) 

aro HplivprpH nnnn rp n 1 1 p c t parh vpar 

CMC UCIIVCICU upull ICljUCjl CuLII yccii. 


7.3.7.5, 
7.4.9.6 


Delivery of the printed End-of-Course 
test administration manuals and ancillary 
materials to districts for the paper-based 
and computer-based administrations. 


C 


Prior to each fall 
administration (see note in 
7.4.1 regarding the 
possibility of 2 manuals 
for 2014-15 school year) 


Usually 70 calendar days before each administration. 


7.4.1 


Delivery of the printed ELA/L and 
Mathematics test administration manuals 
and ancillary materials to districts for the 
paper-based and computer-based 
administrations. 


C 


Prior to each spring 
administration 


Usually 70 calendar days before each administration. 


7.4.1 


Delivery of the printed ELA/L Retake test 
administration manuals and ancillary 


C 


Prior to each fall 
administration (beginning 


Usually 70 calendar days before each administration. 


7.4.1 



ITN Ref. 
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Task Activity 


Critical (C)or 
Extra Critical 


Frequency 


Delivery Notes (Approximations) 


ITN Ref. 


1 1 lu LCI ICII 3 LU UlbLIILLb IUI LIIC UdUCl Ud jCU 

and computer-based administrations. 




in fall ?ni ^\ 

II 1 lull L\J ±J) 






U cllvc l y U 1 LI lc [J I II 1 LcU Lcb L 1 1 Id Lcl Id lb LU 

districts. 


yr 

M 


r 1 1 u i lu c d L 1 1 

administration. 


UbUdlly J WCCro UclUlc CdLII dUlllllllbLldUUIl. 


7 ZL 1 

/ .4-. 1 


Delivery of the preidentification labels to 
districts. 


XC 


Prior to each 
administration 


Usually 3 weeks before each administration. 


7.4.1 


Delivery of the computer-based tests to 
districts. 


XC 


Prior to each 
administration 


The first day of the test administration window, unless 
the system requires pre-loading the assessments. (At 
that point, they should be delivered in sufficient time 
for the schools to load the test prior to the 
administration.) 


7.4.9 


Delivery of test results to districts. 

Results Delivery (RD) details for 1-7 are 
provided in Table 7.5.5. This includes 
print and electronic deliveries. 


RDl- XC 
RD2-XC 
RD3-XC 
RD4-XC 
RD5- C 
RD6- C 
RD7- C 


After each administration 


Varies by results release deliverable. Initial results 

delivered as early as two weeks after the end of the 

administration. 

For EOC administrations: 

• RD1-RD3 (state, district, and school reports) 
must be delivered within two weeks of the 
last day of the administration window, per 
statute. 

• RD4 (hard copies of student results) must be 
delivered within 1 week of RD1-3. 

• RD5 (late reporting) should be completed 
within 2 months of RD4. 

• RD7 (demographic reports) will be delivered 
in August/September for the previous school 
year. 

For E LA/L and Mathematics (Spring): 


7.5.5 
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Task Activity 


Critical (C)or 
Extra Critical 


Frequency 


Delivery Notes (Approximations) 


ITN Ref. 








• RD1-RD3 (state, district, and school reports) 
delivered between the end of April and the 
middle of May. 

• RD4 (hard copies of student results) must be 
delivered before the week of J une 8, per 
statute. 

• RD5 (late reporting) should be completed 
within 3 months of RD4. 

• RD6 (writing image media) is usually 
delivered mid-J uly. 

• RD7 (demographic reports) will be delivered 
in August/September for the previous school 
year. 
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APPENDIX C 

FLORIDA PUBLIC SCHOOL STUDENT PARTICIPATION ESTIMATES, DISTRICT NUMBERS, AND 

SPECIAL DISTRICT/SCHOOL NUMBERS 

STUDENT PARTICIPATION ESTIMATES 

Table C.l indicates the anticipated number of students that would be tested in each grade-level test administration. 
The number of students expected to participate in ELA/L Retake administrations (Table C.2) will be calculated as a 
percentage of the grade 10 population. 

Table C.3 indicates the expected test takers for each EOC Assessment. This calculation assumes 200,000 new 
students for each spring cohort. (Cohort, for the purposes of this ITN, refers to a group of students that enters grade 
9 and is enrolled at the high school for a total of four years.) 

Table C.l Anticipated Number of Students Participating in Assessment Administrations 



Table C.l Anticipated 


Number of Students Participating in Assessment Administrations 




Grade 3 


Grade 4 


Grade 5 


Grade 6 


Grade 7 


Grade 8 


Grade 9 


Grade 
10 


Grade 
11 


2013-14 


212,774 


201,371 


198,642 


203,053 


210,018 


211,085 


222,533 


206,675 


198,131 


2014-15 


217,735 


202,712 


203,183 


202,436 


206,962 


213,634 


228,413 


216,413 


202,282 


2015-16 


225,524 


207,553 


204,609 


207,154 


206,010 


210,031 


229,570 


218,707 


205,215 


2016-17 


223,753 


215,105 


209,564 


208,854 


210,865 


209,364 


226,066 


219,248 


206,493 


2017-181 


224,786 


213,553 


217,105 


213,364 


212,354 


214,365 


225,364 


216,066 


207,248 


2018-191 


225,132 


214,786 


215,553 


221,105 


216,864 


215,854 


230,365 


215,364 


206,707 


2019-201 


224,893 


215,132 


216,734 


219,654 


224,634 


220,268 


231,467 


220,540 


203,211 


1 Optional 


Renewal Period 



Table C.2 Anticipated Number of Students Participating in ELA/L Retake Administrations 



Table C.2 Antic ipatec 


1 Number of Students Participating in ELA/L 
Retake Administrations 




Fall Retake 


Spring Retake 


ELA/L 


65% 1 


45% 1 


: As a percent of anticipated Grade 10 enrollment for applicable years 
from Table C.l. 
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Table C.3 Anticipated Number of Students Participating in EOC Administrations 



Table C.3 Anticipated Number of Students Participating in EOC Administrations 


School 
Year 




Aigeora i 


Aigeora i 


ueometry 


TrtTA 1 

1 U 1 AL 


2014-15 


Spring 


200,000 


200,000 


200,000 


600,000 


Summer 


100,000 


50,000 


50,000 


200,000 


2015-16 


Fall 


75,000 


12,500 


12,500 


100,000 


Winter 


56,250 


3,125 


3,125 


62,500 


Spring 

i j 


242,188 


200,781 


200,781 


643,750 


Summer 


131,641 


50,195 


50,195 


232,031 


2016-1/ 


Fall 


98,730 


12,549 


12,549 


123,828 


Winter 


74,048 


3,137 


3,137 


80,322 


Spring 


255,536 


200,784 


200,784 


657,104 


Summer 


141,652 


50,196 


50,196 


242,044 


■ini 7 i oi 


Fall 


106,239 


12,549 


12,549 


131,337 


Winter 


79,679 


3,137 


3,137 


85,954 


Spring 


259,759 


200,784 


200,784 


661,328 


Summer 


144,820 


50,196 


50,196 


245,212 


T01 Q 1Q1 


Fall 


106,239 


12,549 


12,549 


131,337 


Winter 


79,679 


3,137 


3,137 


85,954 


Spring 


259,759 


200,784 


200,784 


661,328 


Summer 


144,820 


50,196 


50,196 


245,212 


2019-20 1 


Fall 


106,239 


12,549 


12,549 


131,337 


Winter 


79,679 


3,137 


3,137 


85,954 


Spring 


259,759 


200,784 


200,784 


661,328 


Summer 


144,820 


50,196 


50,196 


245,212 


1 Optional Renewal Period 
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DISTRICT NUMBERS AND SPECIAL DISTRICT/SCHOOL NUMBERS 

NOTE: The districts/schools participating in each test administration may vary. For example, some administer only 
the tests required for graduation. 

01-Alachua through 67-Washington 

68- Florida School for the Deaf and the Blind (FSDB) 
68-0011 Deaf Elementary School 

68-0014 Blind Elementary School 
68-0012 Deaf Middle School 
68-0015 Blind Middle School 
68-0013 Deaf High School 
68-0016 Blind High School 

68-0019 Career Education and Transition High School 

69- 0011 Okeechobee Youth Development Center 

71- Florida Virtual Schools 

71-0300 FLVS Full-Time K-8 
71-0400 FLVS Full-Time 9-12 
71-0500 FLVS Part-Time 6-8 
71-0600 FLVS Part-Time 9-12 

71- 0700 FLVS Part-Time K-5 

72- 75 University Laboratory (Lab) Schools 

72- 0011 A.D. Henderson University School & FAU High School 

72- 0020 FAU/SLCSD Palm Pointe Educational Research School @ Tradition 

73- 0341 Florida State University School (FSUS) 

73- 0351 The Pembroke Pines Florida 

74- 0351 Florida A&M University (FAMU) Developmental Research School 

75- 0391 University of Florida (UF) Lab School 

P.K. Yonge Developmental Research School 

80- State Colleges with Adult High School Programs 
80-0005 Daytona State College (Volusia) 
80-0007 Florida State College atj acksonville (Duval) 
80-0011 Indian River State College (St. Lucie) 
80-0020 Pensacola State College (Escambia) 
80-0025 Seminole State College (Seminole) 
80-0026 South Florida State College (Highlands) 

97 - Office of Independent Education (Florida Tax Credit Scholarship students) 

98 - Ahfachkee School 

Special Programs (The following special programs are designations within each district. The "XX" is replaced by the 
two-digit district number.) 

XX-3518 - McKay Scholarship Students 

XX-9998 - Home Education Program Students 
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APPENDIX D 
SAMPLE TEST DESIGN 



Table D.l below is provided as an example of one possible test design that may meet the criteria described in 
Section 7.3.0. Respondents may propose other designs, and a final design will be discussed in negotiation and 
agreed upon between the Department and the selected contractor. 



Table D.l Example Testing Sessions - Grades 3-8 and High School 



Table D.l Example Testing Sessions - Grades 3-8 and High School 


Content Area 


Specification 


Session 1 


Session 2 


Session 3 


English Language 
Arts/Literacy 


Time 


80 minutes 


80 minutes 


80 minutes 


Item Types 


Multiple Choice and 
Short Constructed 
Response 


Multiple Choice 
and Short 
Constructed 
Response 


Extended Writing in 
Response to Text 


Mathematics 


Time 


90 minutes 


90 minutes 




Item Types 


Multiple Choice, 
G ridded/Fill-in 
Response, Short 
Constructed 
Response, and 
1-2 Extended 
Performance Tasks 


Multiple Choice, 
G ridded/Fill-in 
Response, Short 
Constructed 
Response, and 
1-2 Extended 
Performance 
Tasks 
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APPENDIX E 

PRODUCT SPECIFICATIONS AND DISTRIBUTION REQUIREMENTS 

Appendix E is organized as follows: 

Parti perational Test Materials P roduct Specifications & Distribution Requirements 
P art 2 Florida Department of Education PDF Guidelines forSection 508/ADA Documents 
P art 3 Ancillary Materials ShipmentSchedule & Print Quantities 

APPENDIX E, Parti 
OPERATIONAL TEST MATERIALS 
PRODUCT SPECIFICATIONS AND DISTRIBUTION REQUIREMENTS 

Test Documents - General Considerations 

The contractor is responsible for producing camera-ready forms of these tests, printing the test documents, 
and distributing them to school districts. The following requirements apply to all test documents: 

• A method to collect demographic information and other special information (e.g., invalidation) 

o Preferred: Scannable document covers include the student demographic grids and other special 
coding sections. 

• A method to track and log secure materials 

o Preferred: Unique security barcode numbers will be printed on each book containing test questions. 

• Large Printand Braille (contracted and uncontracted) version of the test. Copies of the regularbook (the same 
form) must accompany the shipment of the large printand Braille materials. Braille notes also accompany the 
Braille versions. 

o Preferred: One form of each test (usually form 1) is produced in large print (minimum of 18 point 
type on 14" x 17" approved paper) and in Braille. 

• Forms will be spiraled for random distribution. 

• Documents will be shrink-wrapped in packs of 5 and 25. Quantities smaller than 25 may be desirable and will 
be approved by the Departmentwhen the documentsize makes lifting and packing difficultor hazardous. 

• Test documents must be high-quality and durable 

o P referred: Test documents are 8 %" x 11," saddle stitched or bound; 60# white opaque cougar or 
approved equivalent paper will be used for answer documents and interior pages of test books. 

• Test documents should have high-quality, attractive printing and design 

o Preferred: Covers are printed in one color of ink plus black; Interior pages are printed in one color, 
some with both scannable and non-scannable ink; Color coding or other identification marks are 
included on the spine of the documents to identify them when stacked. 

Test Documents - Specific Considerations: 

1. Grades 3-11 ELA/L Test Documents 

• Up to 40 forms of scannable, sealed testand answer and/or answer documents 

• Number of document types may vary if different portions of a test are in separate documents (e.g., 
writing folders) 

• 1 Reading Passage Booklet as a CBT test accommodation. 

• 1 standalone CBT Worksheet for each CBT tester. 

2. Grades 3-8 Mathematics Documents 

• Up to 40 forms of scannable, sealed testand answer and/or answer documents 

• Number of document types may vary if different portions of a test are in separate documents (e.g., 
performance tasks) 

• 1 standalone CBT Work Folder for each CBT tester. 

• One standalone reference sheet per student, as applicable 
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3. Grade 10 ELA/L Retake 

• P referably, one form of a scannable, sealed test and answer and/or answer document 

• Number of document types may vary if different portions of a test are in separate documents (e.g., 
performance tasks) 

• 1 Reading Passage Booklet as a CBT test accommodation. 

• 1 standalone CBT Worksheet for each CBT tester. 

4. End-of-Course Tests 

• 1 paper accommodation form of the Algebra 1 End-of-Course test and answer book (scannable, 
with an outside seal). 

• 1 paper accommodation form of the Geometry End-of-Course test and answer book (scannable, 
with an outside seal). 

• 1 paper accommodation form of the Algebra 2 End-of-Course test and answer book (scannable, 
with an outside seal). 

• 1 standalone CBT Work Folder for each CBT tester. 

• 1 standalone reference sheet for each Algebra 1, Geometry, and Algebra 2 tester. 

5. Other Accommodations 

• 1 large print paper accommodation form for each test 

• 1 Braille form divided into two volumes for each test 

• 1 one-item-per-page accommodation form for each test 

6. Accommodation Standalones 

• 1 large print reference standalone sheet to accompany each applicable assessmentform 

• 1 large print response grid instructions standalone sheet to accompany each applicable assessment 
form 

• 6 pages of blank Braille paperto accompany each Braille assessmentform 

• 1 large print form of the CBT work folder for each CBT Mathematics assessment 

• 1 large print form of the CBT worksheet for each CBT ELA/L assessment 



Shipment Schedule and Print Quantities for ELA/L and Mathematics Assessments 

• All test materials are shipped in TM2 

• All test documents may contain up to 64 pages 

• All grades 3-11 tests will have up to 40 test forms 

• Grade 10 ELA/L Retake will have one test form 

• Schools will receive one test form (with standalones, as applicable) per preidentified student 

• Districts will receive 15% overage of all test materials 

• For paper-based administration subjects, 10% extra test material is produced 

Shipment Schedule and Print Quantities for EOC Assessments 

• All test materials are shipped in TM2 

• All test documents may contain up to 64 pages 

• Spring EOC tests will have up to 4 unique test forms 

• Fall, Winter, and Summer EOC tests will have one testform each 

• Schools will receive one testform (with standalones, as applicable) per preidentified student 

• Districts will receive 15% overage of all test materials 

• For paper-based administration subjects, 10% extra test material is produced 
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APPENDIX E, Part 2 



FLORIDA DEPARTMENTOF EDUCATION 
PDF GUIDELINES FOR SECTION 508/ADA DOCUMENTS 

The following guidelines may be revised throughout the contract to reflect changes in technology or requirements. 
The contractor will be notified of any changes, as appropriate. 

In order for PDF documents to be posted on the DOE website or any site that is affiliated with the DOE, they must 
pass the Full Accessibility check without error in Adobe Acrobat 9.0 orX. 

DOE uses Adobe Acrobat 9.0 and X to check accessibility of PDF documents. Currently, Adobe Acrobat 8.0 has 
known issues with the accessibility checker. Although a document may pass the full check, this does not necessarily 
mean the document is compliant according to DOE Web standards. The following additional checks are required: 

1. PDF documents must have the correct reading order. The file must be able to be read by assistive technology in 
a logical manner. This can be checked in several ways, which include the following items. 

DOE uses at least two of the following to verify reading order: 

a. Saving the file as an accessible .txtfile and reading it to be sure it is correct 

b. Using the "reading order tool" in Adobe Acrobat 

c. If proficient using a "screen reader" (not the Adobe Acrobat "read out loud"), listen to the document 

d. Using the Reflow view in Adobe Acrobat 

e. Using the "Order" navigation panel 

2. The document must have correct Tab order. 

Tab order refers to how a user can "tab" through the content. The user should be able to tab through the 
document in the order it is intended to be read. If the tab order is incorrect, the assistive technology user may 
jump from one page to another and back and not realize it. Another example of incorrect tab order is the 
practice of structuring tables or columned lists using tabs. 

3. The document must have appropriate "alt text" on all images that have meaning. (Alt text means alternative text 
that appropriately describes the image for an assistive technology user.) 

a. Graphs, charts, flowcharts, etc., cannot simply have the word "graph" in alt text. It must contain enough 
descriptive information for an unsighted user. 

b. G raphics that do not have meaning need to be tagged as "artifacts" or "backgrounds." If the primary purpose 
of an image is decoration or if describing an image will take the listener out of the testing experience and 
serve more as a distraction to the task at hand, then it would fall under this rule. During regular production 
rounds, TDC content leads will determine which elements will be tagged as background. 

c. Putting a blank space for the "alt text" area of an image in an attempt to get the screen reader to skip it is 
not acceptable in "most" cases, as a screen reader will still say the word "graphic." Tagging graphics that 
have no meaning as artifacts or background will cause the screen reader to ignore them. 

d. Graphs and charts need to be described completely to convey the same information to a listener as the 
sighted user would get. If the graph is explained in detail in the "content text" of the document, you can refer 
to that content in the alternate text. For example, if right below a pie chart is a paragraph explaining the 
chart, then the "alt text" on the chart could simply state "Pie Graph which is explained in detail below." This 
is very helpful if the person converting the file to PDF is not the one who created it. 

e. Alternate text should not be provided on anything except for image tags. 

f. Links are not "clickable" in alternative text; therefore, including links in alternative text should be avoided. 
Permission must be obtained from the DOE for special circumstances, such as the approach for tagging the 
Periodic Table of the Elements in Interpretive Products. 

4. If the State Seal, or any other image, is in the header and footer of a document, the first occurrence of the 
header and footer needs to be tagged and readable by a screen reader. All other occurrences should be tagged 
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as artifacts or backgrounds. This will let the listener know that there is at least one state seal (or other image) 
present, but does not repeat on every page that it appears. This rule also applies to headers and footers that 
are repeated on each page of a document. Only new information in the header and footer selections, such as 
page numbers, should be tagged. 

5. Page numbers need to be tagged in such a way that the screen reader reads them. This is helpful in navigating 
the document. 

6. Tables must be tagged as tables, so they are read correctly. A common problem with tables is that they are 
being tagged as paragraphs. 

Tables and table components will be tagged as such: Table headers will have the <TH> tags, table data will 
have <TD> tags , and table rows will have <TR> tags. The proper order for tagging a table is as follows: 
<Table>, <TR>, and <TH>; or <Table>, <TR>, <TD>. The header section of a complex table (i.e, a table with 
more than one header row) should begin with <THead>, and the body section should begin with <TBody>. To 
check a table, compare the layout in the normal view against the layout in the View menu and select Reflow. If 
the layout stays the same and does not shift, the table is properly tagged. 

7. Documents greater than 10 pages in length should contain a clickable table of contents, except when it is not 
feasible. The table of contents should contain the correct tag structure using <TOC> and <TOCI> tags. For 
documents that contain a table of contents that is not clickable, the table of contents needs to be made into 
Bookmarks in the P DF and each table of contents listing needs to be made clickable within the table of contents. 
The Bookmarks Panel should appear on the left side of the screen for documents that do not contain a table of 
contents but are greater than 10 pages in length. This Bookmarks Panel will act as the document's table of 
contents. During regular production rounds, TDC content leads will determine which elements in TDC 
documents will be bookmarked. 

This is needed not only for easy navigation for those using assistive technologies, but it is a convenience to 
sighted visitors as well, since it allows them to click on an item in the table of contents and be directed 
immediately to the content of the document to which it refers. 

8. Content headings must be tagged as headers and not tagged as paragraphs. 

Headings need to be tagged with <H> tags (or Heading 1 <H1>, Heading 2 <H2>, etc.). This helps the listener 
know what subject matter is contained beneath the header and allows for easier navigation. 

9. Files must be saved (in a reduced file size) to Adobe Acrobat version 7.0 with "Fast Web View" enabled for 
documents tagged in Adobe Acrobat 9.0 orX. (The DOE has discovered that converting files created in Adobe 
Acrobat 9.0 to Adobe Acrobat 5.0 increases rather than decreases the file size, and converting files created in 
Adobe Acrobat X to Adobe Acrobat 6.0 or earlier versions distorts the font in interpretive products, so this should 
be avoided.) Reducing the file size ensures that users who have older versions of Acrobat are able to access 
the document. This also decreases the file size for quicker download. 

10. Links must be functional. 

a. Hyperlinks must not be broken and must link to the correct page. This includes e-mail addresses that are 
clickable. 

b. If a link is split between two lines, both lines need to be linked correctly. 

c. Hyperlinks need to be standard, which means in blue font and underlined. This allows for sighted users to 
quickly locate links within the document. 

11. Color should not inhibit the readability of text, and color alone is not to be used to convey meaning. Some 
assistive technologies do not reproduce colored text, and users who are colorblind often cannot differentiate 
every color. 

a. If color is used to convey meaning for sighted users, add a symbol in front of the colored items as well, ag 
this symbol to indicate the meaning of the colored item. 
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b. Under certain DOE Web Services approved circumstances, the color-coded information may be described 
using white text that is not viewable to the sighted user. This "invisible" information must be tagged so that it 
is read prior to the colored text 

12. The following information needs to be noted in the Document Properties: 

Title-Users can setup a screen readerto read the titles instead of the long file names. 

Subject- Specific to the document, and may or may not be the same as the document title. 

Keywords-As with other Web documents, these words are used for search engines and need to be specific to 
the document's contents. 

Language-(Advanced tab) Specified to indicate the primary language of the document. The individual tag 
properties need to indicate text within a document that may be of a different language (e.g., paragraph in 
Spanish needs to specify "Spanish" and paragraph in Haitian Creole needs to specify "French"). 

Author-At this time, the DOE is not requiring the inclusion of "Author;" however, if the name of the author is 
pertinent to the overall document, it is suggested that a name be included. 

13. Lists should be tagged correctly and not tagged as text. The correct order options for list tags are as follows: 
<L>, <LI>, <LBody>; <L>, <LI>, <Lbl>, <LBody>; or <L>, <LI>, <LI_Label><LI_Title>. If lists have lists within list 
items, they need to be tagged correctly. Always tag multiple-choice answer choices/options as lists. Sample 
multiple-choice items appear in Sample Test Materials and in Item Specification books. 

14. Forms that need to be filled out should contain the appropriate form fields and check boxes so users may fill 
them out on the Web. Examples of forms should be indicated as such and do not need to contain form fields 
and check boxes, because they are samples. The answer grids for sample testing materials are not considered 
forms at this time for feasibility reasons. Form field properties need to be filled out so that the information that 
needs to be entered into the form fields is clear. (The "Tooltip" in the form's properties is the portion of the tag 
that is read by the screen reader.) If two form fields have the same "Name" in a document, then both fields will 
populate when one of them is filled out; therefore, each form field must have a unique name in the form field's 
properties. When tagging form fields, it is important to check the accuracy of the Tab order (see list item 2). 

15. Pages cannot be blank. They must contain some text, such as "Notes" in the header or the disclaimer, 'This 
page is intentionally blank." 

16. Pages with content in landscape format should be presented in landscape format, so users may read the content 
without having to print the page. In other words, the Pearson contractor shall rotate landscaped pages inserted 
sideways so the content appears right side up to the viewer. 

17. Words set in bold, underline, or italics may not be used alone to convey meaning. For example, a word set in 
italics cannot have a coded meaning such as "italics= story title." 

a. Add a symbol in front of the italicized, bolded, or underlined information, and tag the symbol to indicate the 
meaning italics, holding, or underlining should convey. 

b. Under certain DOE Web Services approved circumstances, the italicized, bolded, or underlined information 
may be described using white text that is not viewable to the sighted user. This "invisible" information must 
be tagged so that it is read prior to the colored text. 

18. No text should be entered in the "Actual Text" field in the Touchup Properties box. If that is done, some readers 
will only read the actual text and then stop instead of reading the entire document. 

Note: DOE PDF accessibility guidelines were initially effective September 11, 2007. They were revised by DOE Web 
Services to include Adobe Acrobat 9.0 on J anuary 5, 2010, and by DOE Editorial staff on February 17, 2010, for 
added clarifications and forthe purposes of this document. Some DOE staff are now using Adobe AcrobatX, and the 
Department is currently in the process of officially updating PDF accessibility guidelines to include Adobe Acrobat X 
(3/27/12). 
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APPENDIX E, PART 3 - Ancillary Materials Shipment Schedule & Print Quantities 



Product 


Shipment 


Pages 


Number of Copies 


State 


District 


Schools 


Electronic 
Delivery 
(508/ADA 
compliant) 


Test Administration Manual 


TMl 


Up to 400 


25 


10% overage 


1/20 students 


Yes 


Student PrelD Labels (Wave 1) 


TM2 


M /A 

N/A 


N/A 


N/A 


1/student 




Student PrelD Rosters (Wave 1) 


TM2 


M /A 

N/A 


N/A 


N/A 


1 per subject 




Document Count Forms (Preidentified) 


TM2 


N/A 


N/A 


N/A 


2 per 

c 1 1 hiorWn ra Ha 
bUUJcLiyyiaUc 




DocumentCountForms (Non- 
Preidentified) 


TM2 


N/A 


N/A 


N/A 


5 per 

subject/grade 




Online CommentForms 


N/A 


N/A 


N/A 


N/A 


N/A 


Yes 


Security Checklist 


N/A 


variable 


u 


i 
i 


i 
i 


I cb 


District Packing List 


TM1.TM2, 
TM3 


variable 


u 


i 
i 


n 


I cb 


School Packing List 


TM1.TM2, 
TM3 


variable 




i 
i 


i 
i 


Yoc 


District Materials Return Kit 


TM2 


N/A 


1 


1 







School Materials Return Kit 


TM2 


N/A 


1 





1 




Return Materials (e.g., envelopes, 
boxes, special formats) 


TM2 


N/A 





As necessary 


As necessary 




Bills of Lading 




N/A 


o 


Ac nprpc.carv 


Ac nprpc.carv 




Paper Bands 


TM2 


N/A 


10 


15% over 


1/20 student 
documents 




Student PrelD Labels (Wave 2) 
(Spring ELA/L and Mathematics only) 


TM3 


N/A 








1/student (As 
necessary) 




Student PrelD Rosters (Wave 2) 
(Spring ELA/L and Mathematics only) 


TM3 


N/A 








1/subject (As 
necessary) 




Misc. Memoranda 




variable 




As necessary 


As necessary 
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APPENDIX F 

TEST SECURITY REQUIREMENTS, STATUTE, AND RULE 

Chapter 1008.24 of Florida Statutes and Florida State Board of Education Rule 6A-10.042 establish the requirement 
that Florida Department of Education tests are to be maintained in a secure manner during development, 
administration, and scoring in order to preserve the integrity of the tests. When not in use, all test materials are to be 
kept in secure, locked storage. Individuals who have access to secure test materials are not to copy or otherwise 
reproduce test questions or reveal test questions verbally or in writing. Persons who are involved in administering or 
proctoring the tests or preparing examinees for the tests are not to participate in, direct, aid, counsel, assist in, or 
encourage any activity which could result in the inaccurate measurement or reporting of the examinees' 
achievement. Examinees' answers to questions are not to be interfered with in anyway by persons administering or 
scoring the tests. Persons violating test security requirements are guilty of a first degree misdemeanor, punishable 
by a fine of not more than $1,000.00 or imprisonment for not more than 90 days, or both. 

The security requirements and penalties established by the rule and statute must be provided by the contractor to 
each person who has access to tests or test questions during the development, printing, administration, or 
scoring of the tests. 

A copy of the Statute and Rule is part of this appendix. 
Restrictions on Printing for Security Purposes 

The prospective contractors should indicate their ability to comply with the following conditions relative to printing of 
the required tests. These conditions are necessary as a means of maintaining test security. Lack of compliance with 
these conditions may subject the reply to rejection if the Department determines it is in its best interest to do so. 

1. All test negatives and plates must be maintained under lock and key by the printing supervisor. 

2. Unauthorized personnel must not be permitted access to the test negatives, plates, or copies. 

3. All plates and negatives must be destroyed by the contractor upon completion of this contract. 

4. The Department reserves the right to conduct on-site spot checks of the printing processes. 

5. All press pull-ins, trim, and waste material must be shredded at the end of each day's press run by a person 
authorized to do so by the contractor. 

6. Each production run mustbe made underdose supervision of the printing supervisor. 



THE REMAINDER OF PAGE INTENTIONALLY LEFT BLANK 
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FLORIDA TEST SECURITY STATUTE 



1008.24 Test administration and security.— 

(1) A person may not knowingly and willfully violate test security rules adopted by the State Board of 
Education for mandatory tests administered by or through the State Board of Education or the Commissioner of 
Education to students, educators, or applicants for certification or administered by school districts pursuant to s. 
1008.22, or, with respect to any such test, knowingly and willfully to: 

(a) G ive examinees access to test questions prior to testing; 

(b) Copy, reproduce, or use in any manner inconsistent with test security rules all or any portion of any 
secure test booklet; 

(c) Coach examinees during testing or alter or interfere with examinees' responses in any way; 

(d) Make answer keys available to examinees; 

(e) Fail to follow security rules for distribution and return of secure test as directed, or fail to account for all 
secure test materials before, during, and after testing; 

(f) Fail to follow test administration directions specified in the test administration manuals; or 

(g) Participate in, direct, aid, counsel, assist in, or encourage any of the acts prohibited in this section. 

(2) A person who violates this section commits a misdemeanor of the first degree, punishable as provided 
ins. 775.082 or s. 775.083. 

(3) A school district may contract with qualified contractors to administer and proctor statewide 
standardized assessments required under s. 1008.22 or assessments associated with Florida approved courses 
under s. 1003.499, as approved by the Department of Education in accordance with rules of the State Board of 
Education. Assessments may be administered or proctored by qualified contractors at sites that meet criteria 
established by rules of the State Board of Education and adopted pursuant to ss. 120.536(1) and 120.54 to 
implement the contracting requirements of this subsection. 

(4) (a) A district school superintendent, a president of a public postsecondary educational institution, or a 
president of a nonpublic postsecondary educational institution shall cooperate with the Commissioner of Education in 
any investigation concerning the administration of a test administered pursuant to state statute or rule. 

(b) The identity of a school or postsecondary educational institution, the personally identifiable information 
of any personnel of any school district or postsecondary educational institution, or any specific allegations of 
misconduct obtained or reported pursuant to an investigation conducted by the Department of Education of a testing 
impropriety are confidential and exempt from s. 119.07(1) and s. 24(a), Art. I of the State Constitution until the 
conclusion of the investigation or until such time as the investigation ceases to be active. For the purpose of this 
paragraph, an investigation shall be deemed concluded upon a finding that no impropriety has occurred, upon the 
conclusion of any resulting preliminary investigation pursuant to s. 1012.796, upon the completion of any resulting 
investigation by a law enforcement agency, or upon the referral of the matter to an employer who has the authority to 
take disciplinary action against an individual who is suspected of a testing impropriety. For the purpose of this 
paragraph, an investigation shall be considered active so long as it is ongoing and there is a reasonable, good faith 
anticipation that an administrative finding will be made in the foreseeable future. This paragraph is subject to the 
Open Government Sunset Review Act in accordance with s. 119.15 and shall stand repealed on October 2, 2014, 
unless reviewed and saved from repeal through reenactment by the Legislature. 

(5) Exceptional students with disabilities, as defined in s. 1003.01(3), shall have access to testing sites. 
The Department of Education and each school district shall adopt policies that are necessary to ensure such access. 

History.— s. 370, ch. 2002-387; s. 1, ch. 2009-143; s. 7, ch. 2013-225. 
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Florida Test Security State Board of Education Rule 
6A-10.042 Maintenance of Test Security. 

(1) Tests implemented in accordance with the requirements of Sections 1004.93, 1008.22, 1008.29, 1008.30, 
1012.55, and 1012.56, Florida Statutes, shall be maintained and administered in a secure manner such that the 
integrity of the tests shall be preserved. 

(a) Test questions shall be preserved in a secure manner by individuals who are developing and validating the 
tests. Such individuals shall not reveal in any manner, verbally or in writing, the test questions underdevelopment. 

(b) Tests or individual test questions shall not be revealed, copied, or otherwise reproduced by persons who are 
involved in the administration, proctoring, or scoring of any test. 

(c) Examinees shall not be assisted in answering test questions by any means by persons administering or 
proctoring the administration of any test. 

(d) Examinees' answers to questions shall not be interfered with in any way by persons administering, 
proctoring, or scoring the examinations. 

(e) Examinees shall not be given answer keys by any person. 

(f) Persons who are involved in administering or proctoring the tests or persons who teach or otherwise prepare 
examinees for the tests shall not participate in, direct, aid, counsel, assist in, or encourage any activity which could 
result in the inaccurate measurement or reporting of the examinees' achievement 

(g) Each person who has access to tests or test questions during the development, printing, administration, or 
scoring of the tests shall be informed of specifications for maintaining test security, the provisions in statute and rule 
governing test security, and a description of the penalties for breaches of test security. 

(h) During each test administration, school district and institutional test administration coordinators and 
contractors employing test administrators and proctors shall ensure that required testing procedures are being 
followed at all test administration sites. Officials from the Department are authorized to conduct unannounced 
observations of test administration procedures at any test administration site to ensure that testing procedures are 
being correctly followed. 

(2) Test materials, including all test booklets and other materials containing secure test questions, answer keys, 
and student responses, shall be kept secure and precisely accounted for in accordance with the procedures specified 
in the examination program administration manuals and other communications provided by the Department. Such 
procedures shall include but are not limited to the following: 

(a) All test materials shall be kept in secure, locked storage prior to and after administration of any test. 

(b) All test materials shall be precisely accounted for and written documentation kept by test administrators and 
proctors for each point at which test materials are distributed and returned. 

(c) Any discrepancies noted in the number or serial numbers of testing materials received from contractors shall 
be reported to the Department by designated institutional or school district personnel prior to the administration of the 
test. 

(d) In the event that test materials are determined to be missing while in the possession of an institution or 
school district, designated institutional or school district personnel shall investigate the cause of the discrepancy and 
provide the Department with a report of the investigation within thirty (30) calendar days of the initiation of the 
investigation. At a minimum, the report shall include the nature of the situation, the time and place of occurrence, 
and the names of the persons involved in or witness to the occurrence. Officials from the Department are authorized 
to conduct additional investigations. 

(e) In those cases where the responsibility for secure destruction of certain test materials is assigned by the 
Department to designated institutional or school district personnel, the responsible institutional or school district 
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representative shall certify in writing that such destruction was accomplished in a secure manner. 

(f) In those cases where test materials are permitted by the Department to be maintained in an institution or 
school district, the test materials shall be maintained in a secure manner as specified in the instructions provided by 
the Department. Access to the materials shall be limited to the individuals and purposes specified by the 
Department. 

(3) In those situations where an employee of the educational institution, school district, or contractor, or an 
employee of the Department suspects a student of cheating on a test or suspects other violations of the provisions of 
this rule, a report shall be made to the department or test support contractor, as specified in the test administration 
procedures, within ten (10) calendar days. The report shall include a description of the incident, the names of the 
persons involved in or witness to the incident, and other information as appropriate. Officials from the Department 
are authorized to conduct additional investigations. 

(4) Violations of test security provisions shall be subject to penalties provided in statute and State Board Rules. 
Specific Authority 1001.02(1), 1008(24)(1) FS. Law Implemented 1001.02, 1008.24 FS. History-New 7-5-87, Amended 10-26-94. 
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APPENDIX G 
TEST ADMINISTRATION MANUAL WORKFLOW 

The production stages for test administration manuals are defined as follows and are first defined for the Department and 
then defined for the contractor. This is provided to help the respondent anticipate the level of work required on the review 
rounds: 



Table G.l Test Administration Manual Workflow for FDOE and Contractor for Developing Florida-specific 
Standalone Products 



Table G.l Test Administration Manual Workflow for FDOE and Contractor 






FDOE 


Identify which prior administrations' TAMs to use as a starting point for each section of the 
current manual. 


Manuscript 
(MS) Round 


Identify high-level changes, additions, and omissions to the scripts and other sections to be 
included. 




Introduce known or Dotential Dolicv uDdates 

II IU UuUVV l\l IV IV 1 1 V/l KJ \J IX> 1 IUUI KJ \J 1 1 Vw. y UFJUU VV— kj ■ 




Verify accuracy of all FDOE policies outlined in TAM; apply content; policy updates, where 
necessary. 




Review all TAM text by comparing current TAM to most recent and most relevant prior 
TAM(s); verify that relevant text from each section of the prior TAM is reflected in current 
TAM. 




Identify any new naming conventions that need to be incorporated (e.g., changing 
"calculator" to "approved four-function calculator"); note any global edits. 


FDOE 


Verify accuracy of links, email addresses, phone numbers, statutes, dates, and schedules. 


Round 
1 


Review instructions in Test Administrator (TA), School Coordinator (SC), and Technology 
Coordinator (TC) sections to verify consistency (e.g., items packed in DAC box). 


Confirm TAM accurately indicates which ancillaries will be posted; verify file format(s). 




Add queries for contractor program team in green text boxes. 




Include placeholders (text in purple text box) in sections to be addressed at Round 2 
(e.g., outstanding policy decisions). 




File clean-up before returning TAM to contractor: 
Change internal queries from orange to purple. 
Remove FDOE internal queries where possible. 
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Round 


Task 




Confirm that all Round 1 edits have been applied. 




Confirm that queries to contractor from Round 1 have been addressed. 




Replace all placeholders with content. 




For assessments with CBT & PBT TAMs: 

Review instructions in PBT TAM against instructions in CBT TAM for consistency. 


FDOE 


Compare C BT and/or P BT test form to TAM to 

o Verify accuracy of scripts (incl. electronic practice test scripts), 

o Verify instructions, 

o Verify seal configurations, 

r> \/prif\/ nanp rpfprpnrpc. 
\j veiny [jayc i cici ci ilco, 

o Verify titles, and 

n Vprifv tool Hp<;rrintinn c ; 


Round 
2 


Verify session schedule references in scripts (e.g., Session 2, Day 2); verify session length 
references in script. 




Ensure screen shots and instructions are up-to-date. 




Respond to contractor queries (light blue text boxes); apply checkmarks to approve 
suggested edits, as needed. 




E nsure that contractor program team checks have been made; include reminders/queries 
regarding checks that must be performed; schedule conference call/WebEx, if necessary, to 
discuss questions/concerns regarding Program Team (PT) verification. 




C nnrliiri" rminrltahlp rparl-rhrminh rnmnarinn tn mnlrinlp nrinr TAMc fnr rnncktpnrv 

V- Ul IU UL L I UU I IU Lu UIC ICuU UIIUULjll, LUIIIUGIIIIU LU MIUIUUIC U\ IUI 1 nrl j IUI LUI lo lo LCI ILy , 




Ensure FDOE specialists (e.g., individuals from other offices) verify information (e.g., any 
updates to students to be tested, any updates to ESE accommodation information) 




File clean-up before returning TAM to contractor: 
Change internal queries from orange to purple. 
Remove FDOE internal queries where possible. 
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Confirm that all Round 2 edits have been applied. 




Confirm that queries to contractor from Round 2 have been addressed. 




Verify contractor program team checks have been performed by appropriate program team 
members. 


FDOE 


Check TOC, headers, footers, pagination, and internal page references for accuracy. 


Round 
3 


Perform a cold read of the entire TAM to identify any final text edits to be made. 




Verify all FPOs (placeholders) have been replaced with accurate, up-to-date graphics. 




Verify checklists and ancillaries in appendices for accuracy and consistency (i.e., that forms 
are approved versions). 




Review third-party proofreader's queries/suggested edits; apply checkmarks to approve 
suggested edits or indicate revised text using verbatim text edits in yellow text boxes. 




File clean-up before returning TAM to contractor. 
Remove FDOE internal queries. 





Task 




FDOE 

Final 
Round 


Confirm that all Round 3 edits have been applied. 


Confirm that queries to contractor from Round 3 have been addressed. 


Perform a final visual check of the document. 
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Round 






Draft new TAM using TAM Information Sheet and MS round FDOE markup in previous 
TAM. 




Communicate with FDOE about edits to be applied based on MS round markup. Identify 
version(s) of previous TAM from which language has been pulled if not the approved final 
version (e.g., language in x, y, and z sections of CBT TAM pulled form EOC TAM round 3). 




Include queries to FDOE for sections that require confirmation, revision, or additional 
information. 


Contractor 
Round 
1 


Insert FPO (placeholder) graphics; insert forms/images from contractor P&D program team 
when available and note the following: 

o G raphics should depict information for the applicable admin, such as correct start 
date and logical start time. 

o For CBT: Screenshots should be taken using the correct user profile (e.g., CBT 
Coordinator). 

o Sample students should have age appropriate date-of-birth fields and grade levels 
should match the administration. 




Pprform a rlpan rpad of TAM and in^prt nupripc; whprp tpxt mav nppd to hp undatpd ha<;pd 

1 CI 1 \J 1 1 1 1 U LICU 1 1 ILUU \J \ 1 r\ 1 1 U 1 1 U II IjLI L UULI ILJ VVIICIC LLAL 1 1 1 U V 1 ILLU WJ UL UUUU LC U UU JCU 

on comments at debrief, comment form feedback analysis, call center data, etc.; suggest 
changes; highlight any tentative revisions for FDOE to accept/reject/revise. 




F ile clean-up before posting TAM to F DO E : 
Remove internal queries where possible. 
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Contractor 
Round 
2 



Apply FDOE edits. Ifan FDOE edit seems incorrect or inappropriate, contact FDOE to inquire. 
All edits should be applied unless, during contractor's review, FDOE is consulted and agrees 
to leave out edit. 

Consult appropriate contractor program staff (per Florida P rogram Team org chart) and 
acquire approval/sign-off on relevant script, technical, and P&D sections ofTAM, including a 
review of naming conventions for P&D materials. 

Insert FPO (placeholder) graphics; insert forms/images from contractor P&D program team 
when available and note the following: 

o G raphics should depict information for the applicable admin, such as correct start 
date and logical start time. 

o ForCBT: Screenshots should be taken using the correct user profile (e.g., CBT 
Coordinator). 

o Sample students should have age appropriate date-of-birth fields and grade levels 
should match the administration. 

Perform a role play to check operational scripts against currently available test forms/books 
and verify accuracy of instructions, page references, screen shots, tool descriptions, etc. 

Verify session schedule references in scripts (e.g., Session 2, Day 2); verify session length 
references in script. 

Verify the online management system instructions by intended user role. 

Verify the online management system and computer-based platform screen shots. 

Verify emails, phone numbers, URLs, dates, and schedules. 

Verify which ancillaries will be posted on the online management system, and in what file 
format(s) (check against approved ancillaries matrix). 

Perform a clean read and provide queries/suggested additional edits. Suggested edits should 
be indicated using blue comment boxes for FDOE to approve or reject in the next review 
round (i.e., should not be incorporated into the body of the TAM prior to FDOE approval and 
should not be made using yellow text boxes, which could be mistaken for FDOE verbatim 
edits). 

Indicate content that has and has not yet been verified (e.g., add a text box indicating that 
school box contents have not yet been verified). 

File clean-up before posting TAM to FDOE: 
Remove contractor internal queries. 

Replace FDOE queries, as necessary, and ensure contractor responses are included. 
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Round 


Task 


Pearson 
Round 
3 


Apply FDOE edits. If an FDOE edit seems incorrect or inappropriate, contact FDOE to 
inquire. All edits should be applied unless, during contractor's review, FDOE is consulted and 
agrees to leave out edit. 


Confirm all FPOs have been replaced with updated artwork. 

o G raphics should depict information for the applicable admin, such as correct start 
date and logical start time. 

o For CBT: Screenshots should be taken using the correct user profile (e.g., CBT 
Coordinator). 

o Sample students should have age appropriate date-of-birth fields and grade levels 
should match the administration. 


Verify online management system instructions are accurate and up to date. 


Ensure all program team reviews are complete and that verification is indicated in TAM; where 
suggested edits arise during PT verification, these edits should be placed in blue comment 

hnypQ fnrFnOF annrnual nrinrtn inrnrnnrarinn into tPYt - 

UUACj IUI \U\JL. ouuiuvoi UIIUI LU II ILUI [Jul d Uvl 1 IIIIU LCAL. 


Perform final full review of entire TAM for content, formatting, consistency of information and 
instructions across sections (e.g., TA, SC, and TC have same instructions and understanding 
of responsibilities). 


Consult appropriate contractor program staff (per Florida P rogram Team org chart) and 
acquire approval/sign-off on relevant script, technical, and P&D sections ofTAM. 


Verify TOC, headers/footers, pagination, internal page references. 


File clean-up before posting TAM to FDOE: 
Remove contractor internal queries. 

Replace FDOE queries, as necessary, and ensure contractor responses are included. 



Round 


Task 




Confirm all edits have been applied. 




Confirm that approved versions of all checklists and ancillaries are being used. 


Contractor 
Final 
Round 


Perform a final, full clean read. 


File clean-up before posting TAM to FDOE: 
Remove contractor internal queries. 

Replace FDOE queries, as necessary, and ensure contractor responses are included. 




Confirm that when the final version is posted to FDOE, no marks or comments are included except 
where the copyeditor has made an unrequested edit or where FDOE had a query and contractor is 
providing a response. 
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APPENDIX H 
TEST ACCOMMODATIONS 

Test Accommodations for Students with Disabilities (General Information) 

As specified in this appendix, accommodations may be provided to students with disabilities enrolled in public 
schools with current lEPs or Section 504 plans. Care must be taken to provide a student with only the 
accommodations permitted for that student. In addition, students with disabilities must be provided the opportunity to 
participate in practice activities for the tests with appropriate allowable accommodations. 

Determination of appropriate accommodations in assessment situations for students with disabilities shall be based 
on the individual needs of each student. Decisions on accommodations shall be made by the IEP or Section 504 
team and recorded on the IEP or Section 504 plan. Students with disabilities should be oriented to any test situation 
through test-taking instruction designed to familiarize them with testing format and procedures. This orientation 
should take place near the time of testing. Guidelines recommended for making accommodation decisions include: 

1. Accommodations should facilitate an accurate demonstration of what the student knows or can do. 

2. Accommodations should not provide the student with an unfair advantage or interfere with the validity of a test; 
accommodations must not change the underlying skills that are being measured by the test. 

3. Accommodations must be the same or nearly the same as those needed and used by the student in completing 
classroom instruction and assessment activities. 

4. Accommodations must be necessary for enabling the student to demonstrate knowledge, ability, skill, or 
mastery. 

Students with disabilities who are not currently enrolled in public schools or receiving services through public school 
programs and require accommodations in order to participate in a test administration may have access to 
accommodations if the following information is provided: 

• Evidence that the student has been found eligible as a student with a disability as defined by s. 1003.01(3) (a), 
F.S., or Rule 6A-19.00K6), FAC, and 

• Documentation that the requested accommodations are regularly used for instruction. 

For further information, contact the Bureau of Exceptional Education and Student Services at 850-245-0475 or visit 
the following website: www.fldoe.org/ese/fcatasd.asp . 

Computer-Based Accommodations for Students with Disabilities 

Students with disabilities may participate in CBT administrations using computer-based accommodations as 
indicated on their lEPs or Section 504 plans. CBT accommodations may be included in the category of flexible 
presentation accommodations for CBT administrations. They may be provided to students along with other 
allowable test administration accommodations. 

CBT accommodations must be accessible through the computer-based testing platform that will be used for non- 
accommodated students. Accommodations should be administrator-selectable. 

A. Large Print Font 

A fixed large print font size (minimum 18-point) that is an enlargement of the regular type is available for 
students who require it. Computer interface (scrollbar, buttons, cursor/pointer, etc.) is also enlarged accordingly. 

B. Color Contrast 

Options for variable font color and background color display preselected font and background color combinations 
on the computer screen. S pecific highlighter colors are also provided for students who require them. 

The color combinations may include, but are not limited to, the following: 

• White Text/Black Background (Blue Tracking; G reen Highlighter) 

• Black Text/Beige Background (Green Tracking; Yellow Highlighter) 
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• Black Text/Light Green Background (Yellow Tracking; Orange Highlighter) 

Without the use of color contrast, the screen displays the following: 

• Black Text/White Background (Blue Tracking; Blue Highlighter) 

C. Zoom 

Feature allows students to self-select print size and magnification. Zoom levels include: lx, 1.25x, 1.5x, 1.75x, 
2x, 2.25x, 2.5x, 2.75x, 3x, 3.5x, 4x, 5x, 6x, 7x, 8x, lOx, Computer interface (scrollbar, buttons, cursor/pointer, 
etc.) also zooms accordingly. 

D. Screen Reader 

The screen reader provides text-to-speech audio presentation of directions, test items, and answer choices. The 
Department will provide the contractor with screen reader specifications at the beginning of the contract detailing 
the various requirements and procedures for read-aloud text and technology. 

E. Assistive Devices 

The assistive devices form allows the use of computer-based assistive devices with no other accommodated 
features enabled for students who require assistive devices but do not require other accommodated CBT forms. 

Paper-Based Accommodations for Students with Disabilities 

Students with disabilities may participate in CBT administrations using paper-based accommodations as indicated on 
their lEPs or Section 504 plans. Paper-based accommodations may be included in the category of flexible 
presentation accommodations for CBT administrations and may be provided to students along with other allowable 
test administration accommodations. 

In order to receive paper-based test materials for any CBT administration, the requirement for a paper-based test OR 
one or more of the following conditions should be documented on a student's IEP or Section 504 plan: 

• The student cannot access a computer. 

• The student requires an accommodation that cannot be implemented in conjunction with a computer-based 
administration (e.g., the student requires a specialized setting or uses adaptive furniture that cannot 
accommodate a computer workstation). 

• The student requires an accommodated CBT form that is not available. 

• The student has a flexible scheduling/extended time accommodation that requires testing one session over more 
than one day. 

• The student is hospital/homebound. 

Other conditions, in addition to those listed above, may be considered by the student's IEP or Section 504 plan team. 

A. Regular Print 

A regular print paper version of the test may be requested for a student who requires it. 

B. Braille 

A contracted or uncontracted Braille version may be requested for a student who uses Braille materials. Some 
test items may be altered in format for Braille versions of the test as authorized by the Department. Test items 
that have no application for the Braille reader will be deleted as authorized by the Department. Student 
performance standards that cannot be assessed in the Braille format will be deleted according to the 
requirements of s. 1008.22, F.S. 

C. Large Print 

A paper-based large print version of the test may be requested as a unique accommodation for eligible students 
who are unable to access the computer-based large print or zoom accommodation via the computer-based 
testing platform. 
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D. One-ltem-Per-Page 

One-item-per-page materials may be requested as a unique accommodation for paper-based tests for use by 
students who require one test item on a page, fewer test items on a page, increased space between test items, 
or true black-and-white print. 

E. Reading Passage Booklet 

Eligible students participating in computer-based Reading tests may be provided regular print or large print 
booklets that contain the reading passages but do not contain the test items. Although Reading Passage 
Booklets are included in the category of paper-based accommodations, students who require them will test on 
the computer. 

Administration Accommodations for Students with Disabilities 

Administration accommodations must be provided as indicated on student lEPs or Section 504 plans. Listed below 
are allowable administration accommodations for Florida's statewide assessments. Not all allowable 
accommodations are listed. 

A. Flexible Presentation 

• Paper-based test materials (regular print, braille, large print) and accommodated computer-based test forms 
(large print font, color contrast, zoom, screen reader) may be requested for students who require them. 

• Oral presentation may be provided for all directions and for items/answer choices other than passages and 
items/answer choices in Reading tests. Reading passages, items, and answer choices may not be read 
aloud or presented orally, but must be read by the student through visual or tactile means. 

■ Mathematics, science, and history items/answer choices may be read aloud by a test administrator 
or presented orally via screen reader. 

• Signed presentation may be provided for all directions and for items/answer choices other than passages 
and items/answer choices in Reading tests. Reading passages, items, and answer choices may not be 
signed, but must be read by the student through visual or tactile means. The test administrator may 
translate using the same method of sign language that the student regularly uses in the classroom, but must 
be careful not to use signs that might lead the student to the correct response. In such cases, fingerspelling 
may be used as an alternative. 

• For oral or signed presentation, the test administrator may sign or read aloud directions, items, and answer 
choices, other than passages and items/answer choices in Reading tests, to the student in the manner that 
is regularly used in the classroom. Examples of allowable oral/signed presentation include, but are not 
limited to, reading/signing items to a group of students, reading/signing every item to a student individually, 
reading/signing items only when a student requests. 

• Test directions may be repeated, clarified, or summarized. 

• A student may be provided with a copy of directions from the administration script that is read by the test 
administrator. 

• A student may be allowed to demonstrate that he or she understands the directions (e.g., repeating or 
paraphrasing). 

• A student may read aloud test items and answer choices to him/herself. This would require use of a device 
such as a WhisperP hone®, or would require that the student be tested in a separate setting so that other 
students are not disturbed. 

• Verbal encouragement (e.g., "keep working," "make sure to answer every question") may be used; however, 
it may not be used to cue a student regarding correct/incorrect responses. 

• Students may use magnification devices (e.g., CCTV/video magnifiers, reading loupes, handheld 
magnifiers). Devices must be used without accessing image-upload features (e.g., devices with a memory 
card must have the memory card removed during testing). 

• A student may use a straightedge to maintain or enhance visual attention to test items. 
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• Portions of the test may be masked to direct attention to uncovered item(s). 

• Colored transparencies/overlays may be used. 

• Test books may be secured to a work area if no adhesives are used on scannable documents. If 
necessary, scannable documents may also be secured with adhesives if responses are transcribed into 
replacement documents. 

• Positioning tools, such as a reading stand, may be used. 

• A student may use a highlighter to highlight key words or phrases in directions, passages, test items, and 
answer choices. 

B. Flexible Responding 

• The student may use varied methods to respond to the test, including written, signed, and verbal responses. 
(Written responses may include the use of devices such as the Graphic Aid for Mathematics or the 
geoboard for students using braille.) A test administrator or proctor may transcribe student responses to the 
format required by the test. Recorded responses must accurately reflect the response of the student, 
without addition or enhancement by the test administrator or proctor. 

• A student may dictate responses to a proctor. 

• A student may use speech-to-text technology to record answers. 

• A student may indicate answers to test items using the mouse, keyboard, or assistive technology that uses 
either device. 

• A student may use a computer switch to indicate answers. 

• A student may use a computer/alternative keyboard to indicate answers. 

• A student may use a pointing device to indicate answers. 

• A student may use other communication devices to indicate answers. 

• A student may use mathematics grids/guides to organize mathematical computation. 

• A student may use special paper such as raised line, shaded line, or color-coded (would require that 
responses are then transcribed by school/district personnel). 

• Test administrators may check periodically to be sure that students are entering answer choices correctly on 
the computer or marking in the correct spaces in the test book. 

C. Flexible Scheduling 

• A student may be administered a test during several brief sessions, allowing frequent breaks during the test 
sessions. Between sessions and during breaks (e.g., restroom breaks, lunch breaks), students must be 
closely monitored to ensure that they do not share responses or change responses to items that were 
already completed. 

• A student may use a specific time of day for specific sessions. 

• A student may be provided extended time to complete a test. Extended time must be provided in 
accordance with the student's IEP or Section 504 plan. Extended time is not unlimited time; it should 
align with the accommodation used regularly in the student's classroom instruction and assessments. The 
student is not required to use all of the extended time that is allowed and may end the testing session prior 
to the expiration of the extended time. 

• Other considerations for Scheduling accommodations when administering Florida's statewide assessments 
to students with disabilities include: 

• Each test or test session must be completed within one school day. Students who have a flexible 
scheduling accommodation that allows them to test over more than one day will be required to participate in 
testing using paper-based test materials. 
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• For students using paper-based test materials to test over more than one day, the following conditions must 
be maintained to ensure the validity of the test administration across days: 

■ The student may not be permitted to change his or her responses to items that were completed on a 
previous day. 

■ The test administrator must closely supervise the administration of the test on an individual basis to 
ensure that answers from the previous day are not changed and that the student does not preview parts 
of the test to be answered the following day. 

■ The test administrator must use a paper clip or binder clip to secure the answer documents and prevent 
the student from reviewing his or her answers from the previous day. ALL CLIPS MUST BE 
REMOVED AFTER TESTING. Staples or tape should NOT be used on books containing student 
answers. 

D. Flexible Setting 

• A student may be administered a test in a familiar place with a test proctor present, and/or a familiar person 
who has been appropriately trained may administer the test. Procedures for test security must also be 
followed and precautions taken in order to ensure that the test remains secure. 

• A student may be administered a test individually or in a small group setting. A small group should be of a 
size comparable to the normal instruction group size indicated on the student's IEP or Section 504 plan. 

• If a student requires use of an accommodation that may disturb other students, the student should be tested 
in a separate setting. 

• A student may use special lighting. 

• A student may use adaptive or special furniture. 

• Special acoustics, such as FM systems, may be used to enhance sound or special rooms may be used to 
decrease auditory distractions. 

• The opportunity for movement may be increased or decreased. 

• Stimuli may be reduced (e.g., by limiting the number of items around the student's desk or computer 
station). 

• White noise/sound machines or approved music may be used to reduce auditory distractions. 

E. Assistive Devices and Tools 

• Any student who requires software programs to ensure access to the computer and to approved assistive 
technology (e.g., switch scanning software, specialized on-screen keyboards, mouse emulators) must be 
able to use these devices with the computer-based testing platform. 

• A student may use varied assistive devices to access the computer (e.g., alternative keyboards, trackballs, 
joysticks, switch scanning systems, touch screens, mouth sticks, head wands, eye-gaze or head control 
systems). 

• Visual magnification and auditory amplification devices may be used. 

• Real coins may be used to represent coins depicted in test items. Coins may not be used as counting 
devices. 

• For students with visual impairments, an abacus and products such as the Graphic Aid for Mathematics, 
Math Window®, and the geoboard may be used. These devices are approved substitutes for paper and 
pencil computation. 

• Other assistive technology typically used by the student in classroom instruction may be used provided the 
purpose of the testing is not violated. Implementation of assistive devices must ensure that test responses 
are the independent work of the student. 

• Students who use sign language as their primary means of communication may use an English/sign or 
sign/English translation dictionary. The dictionary must be similar to one used in the instructional setting 
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and may not contain definitions of words. The dictionary may contain the sign picture, the word, synonyms, 
and an index. 

• Manipulative materials, including, but not limited to, counters, base-10 blocks, clock faces, or geometric 
shapes, are not allowed to be used by any student during the test administration. 

■ FOR BRAILLE MATERIALS ONLY: While the use of manipulates is notan allowable accommodation, 
occasionally an item in a Braille test will require the use of manipulatives to give the student a tactile 
reference. If/when such items occur, test administrators will be provided with the instructions allowing 
students to use the manipulatives for those specific test items only. This modification is approved by 
the Department and applies only to students using Braille test materials. 

Unique Accommodations 

In accordance with Rule 6A-1.0943, FAC, school districts may request unique accommodations for individual 
students with disabilities. Unique accommodations usually involve alterations of existing test materials. In addition, 
they must be regularly used by the student for classroom instruction and must not alter the underlying content of the 
test. Each unique accommodation must be approved by the Commissioner of Education or a designee prior to its 
use. Written requests for unique accommodations must be submitted by the district assessment coordinator to the 
Bureau of K-12 Student Assessment at the FDOE. Accommodation requests must be reviewed by district level staff 
before being sent to the FDOE. In addition, the signatures of both the district exceptional student education (ESE) 
coordinator and the district assessment coordinator are required. 

Administration Accommodations for English Language Learners (ELLs) 

Districts are required to offer accommodations to students identified as ELLs. Allowable accommodations for ELLs 
are listed below. The test may be administered with any one of these modifications or a combination of 
accommodations that are determined to be appropriate for the particular needs of ELLs. However, all testing, with or 
without accommodations, must be completed during the prescribed testing dates. 

A. Flexible Setting 

• ELLs may be offered the opportunity to be tested in a separate room with the English for Speakers of 
Other Languages (ESOL) or heritage language teacher acting as test administrator. Parents must be 
informed of this option for students not of legal age and shall be given the opportunity to select the 
preferred method of test administration. 

B. Flexible Scheduling 

• ELLs may take a test session during several brief periods within one school day; however, each test or 
test session must be completed within one school day. 

• ELLs may be provided additional time; however, each test or test session must be completed within one 
school day. 

C. Assistance in Heritage Language 

• ELLs may be provided limited assistance by an ESOL or heritage language teacher using the student's 
heritage language for all directions and for items/answer choices other than passages and items/answer 
choices in Reading tests. This should not be interpreted as permission to provide oral presentation of 
items/answer choices in E nglish or in the student's heritage language. 

■ The ESOL or heritage language teacher may answer student questions about the general test 
directions in their heritage language. If the test is administered to a group of students, the teacher may 
answer questions about directions for the benefit of the group. Questions of clarification from individual 
students must be answered on an individual basis without disturbing other students. 

■ Mathematics, Science, and History items: The teacher may answer specific questions about a word 
or phrase that is confusing the student because of limited English proficiency, but is prohibited from 
giving assistance that will help the student solve problems and answer test items. A student's 
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questions must not be answered in a way that would lead the student to infer the correct answer to an 
item. 

D. Approved Dictionary 

• ELLs must have access to an English-to-heritage language translation dictionary and/or heritage 
language-to-English translation dictionary, such as those made available to ELLs in an instructional 
setting. However, a dictionary providing definitions written exclusively in the heritage language or in 
English may not be provided. 
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APPENDIX I 
DATA PRIVACY & SECURITY POLICY 
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V. ENFORCEMENT 

I. INTRODUCTION 

Personally Identifiable Information ("PH") on students, including information on their performance on the 
assessments and limited demographic information, will be collected by the Contractor. Disclosure of Pll to the 
Departmentand its contractors is authorized by the Family Educational Rights and Privacy Act ("FERPA") forthe 
purposes of- 

(1) Conducting studies, for or on behalf of the Department and Florida's school districts, to develop, validate, 
and administer predictive tests or to improve instruction (34 CFR 99.31(a)(6)(i)(A) & (C)); and 

(2) Assisting in the evaluation of federal- and state-supported education programs and ensuring compliance 
with federal legal requirements related to such programs - in particular related to state-level assessments 
and accountability systems (34 CFR 99.31(a)(3) & 99.35). 

More specifically, the data will be used by the Department or by one or more of the Department's Contractors— 

• to validate, pilot and field test, and improve the assessments; 

• to report assessment results back to the Department and Florida school districts in a form that is useful to 
them; 

• to prepare reports on student performance for the Department and Florida school districts, and the public 
(Pll may not be included in public reports or in reports to states or school districts that were not the source 
of the Pll); 

• to analyze test results for purposes of accountability, including promotion and graduation decisions for 
individual students; teacher and school leader evaluations; school accountability determinations; 
determinations of principal and teacher professional development and support needs; and teaching, 
learning, and program improvement; and 

• to carry out studies designed to improve instruction on behalf of the Departmentand Florida school districts. 

Access to student data remains the legal responsibility of the Department and Florida school districts, in 
accordance with FERPA. 
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This Policy describes, in general, (i) what steps the Department takes to protect Pll that is accessed by or 
provided to the Department or the Contractor; (ii) how that information is used; (iii) with whom the Department 
shares that information, and (iv) the steps the Department takes to protect the security of that information. 

Monitoring of the implementation of this Policy is the shared responsibility of the Departmentand the Contractor. 

Each of the privacy and security provisions in this Policy is effective upon contract execution. 

II. DEFINITIONS 

Capitalized terms referenced herein but not otherwise defined shall have the meanings as set forth below: 

"Authorized User" means an individual Department or Contractor employee authorized by the Department to 
access Pll maintained by the Department or the Department Contractor. 

"Destroy" or "Destruction" means the act of ensuring the Pll cannot be reused or reconstituted in a format which 
could be used as originally intended and that the Pll is virtually impossible to recover or is prohibitively expensive 
to reconstitute in its original format. 

"FERPA" means the Family Educational Rights and Privacy Act of 1974 (codified at 20 U.S.C. § 1232g) and its 
implementing regulations, as they may be amended from time to time. The regulations are issued by the U.S. 
Department of Education, and are available at http://www2.ed.gov/policy/gen/reg/ferpa/index.html. 

"Contractor" means each Department contractor, including any subcontractor(s), that may be required to 
maintain or handle Pll in the course of providing services in support of the Department. 

"Personally Identifiable Information" (or "Pll") means any information defined as personally identifiable 
information under FERPA. 

III. PRIVACY OF PERSONAL INFORMATION 

A. Basic Privacy Protections 

1. Compliance with Law and Policy. All Pll uploaded to or made accessible to the Department or 
Contractor will be handled, processed, stored, transmitted and protected in accordance with all 
applicable federal data privacy and security laws (including FERPA) and with this Policy. 

2. Training. Employees of the Department (including temporary and contract employees) and the 
employees (including temporary and contract employees) of the Contractor are educated and trained 
on the proper uses and disclosures of Pll and the importance of information privacy and security. Such 
training will include training for new employees and refresher training for current employees. 

3. Personnel Guidelines. All the Departmentand Contractor employees are required to be aware of and 
work to protect the confidentiality, privacy, and security of Pll. The Department, the Contractor, and 
their respective personnel shall not access Pll except to comply with a legal obligation under federal or 
state law, regulation, subpoena or action by the Department that requires such access, or if there is 
legitimate need for the information to maintain data systems or to perform required services. The 
following list provides a general description of the internal policies with which the Department, the 
Contractor, and their respective personnel are required to comply: 

- Limit internal access to Pll to the Department and Contractor personnel with proper authorization 
and allow use and/or disclosure internally, when necessary, solely to personnel with a legitimate 
need for the Pll to carry out the educational purposes of the Department. 

- Allow access to Pll by external parties only as authorized by the Department. 

- Require that materials containing Pll in electronic form are stored solely within encrypted data 
repositories and Pll are not available on unencrypted shared drives that are used by other users or 
on a local drive. Pll must be encrypted in transit as well. 
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- When Pll is no longer needed, delete access to Pll, in accordance with secure destruction 
procedures that the Department will provide. 

- Permit the Department and Contractor employees to store or download information onto a local or 
encrypted portable devices or storage only when necessary, and to create a written record for 
retention verifying that the information is encrypted and stored in password-protected files, and that 
devices containing the information have appropriate security settings in place (such as encryption, 
firewall protection, anti-virus software and malware protection). 

- Require that any downloaded materials consisting of Pll remain in the United States. 

- Prohibit the unencrypted transmission of information from the Department, or any other source of 
Pll, wirelessly or across a public network to any third party. 

B. Access to Pll 

1. The Department makes the final determination regarding access to Pll. 

2. Parent Complaints. The Department shall be responsible for handling any complaints from parents (or 
students 18 and over) regarding the disclosure or use or accuracy of Pll. 

3. Privacy Administrators. The Department will designate a privacy administrator who is responsible for 
managing access to Pll maintained by the Department or the Contractor by designating authorized 
users, and for determining the scope of Pll to which they have access. The Privacy Administrator shall 
be responsible for making all administrative decisions regarding access to and use of Pll. 

IV. INFORMATION SECURITY PROGRAM 

The security of the Pll is of critical importance to the Department. The Department's IT Security Program 
consists of technical, physical and administrative safeguards to protect the Pll. The Department's IT Security 
Program is designed to identify, manage and control the risks to system and data availability, integrity, and 
confidentiality, and to ensure accountability for system actions. The Department's Security Program includes, 
and the security program of each Contractor is required to include, the following key general processes which 
may be more fully described in other materials as necessary: 

A. Information Security Risk Assessment 

The Department periodically conducts, and the Contractor is required to periodically conduct, an accurate 
and thorough assessment of the potential risks and vulnerabilities to the confidentiality, integrity, and 
availability of electronic, paper, or other records containing Pll maintained by the Department and the 
Contractor; to report such risks as promptly as possible to the Department or other official within the 
Department designated to be responsible for data privacy and security compliance; and to implement 
security measures sufficient to reduce identified risks and vulnerabilities. Such measures shall be 
implemented based on the level of risks, capabilities, and operating requirements. These measures must 
include as appropriate and reasonable the following safeguards: 

1. Administrative Safeguards 

a. Sanctions: Appropriate sanctions against the Department or contractor personnel who fail to 
comply with the Department security policies and procedures, with the potential for criminal referral 
if warranted. 

b. System Monitoring: Procedures to regularly review records of information systems activity, 
including maintaining access logs, access reports, security incident tracking reports, and periodic 
access audits. 

c. Security Oversight: Assignment of one or more appropriate senior officials within the Department 
and the Contractor, as applicable, to be responsible for developing, implementing, and monitoring 
of safeguards and security issues. 

d. Appropriate Access: Procedures to determine that the access of the Department and the 
Contractor personnel to Pll is appropriate and meets a legitimate need to support their roles in 
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business or educational operations. Procedures for establishing appropriate authorization and 
authentication mechanisms for the Department and the Contractor personnel who have access to 
Pll. 

e. Employee and Contractor Supervision: Procedures for regularly monitoring and supervising the 
Department and Contractor personnel who have access to Pll. 

f. Access Termination: P rocedures for terminating access to P II when employment ends, or when an 
individual no long has a legitimate need for access. 

g. Recording Requests and Disclosures: Disclosures of Pll to, and requests for disclosures of Pll 
from, third parties -- other than employees of the Department or of the Contractor that provided the 
Pll to the Department-- are recorded. 

2. Physical Safeguards 

a. Access to Pll: Procedures that grant access to Pll by establishing, documenting, reviewing, and 
modifying a user's right of access to a workstation, software application/transaction, or process. 

b. Awareness Training: On-going security awareness through training or other means that provide 
the Department and the Contractor personnel (including management) with updates to security 
procedures and policies (including guarding against, detecting, and reporting malicious software). 
Awareness training should also address procedures for monitoring log-in attempts and reporting 
discrepancies, as well as procedures for safeguarding passwords. 

c. Incident Response Plan: Procedures for responding to, documenting, and mitigating where 
practicable suspected or known incidents involving a possible breach of security and their 
outcomes. 

d. Physical Access: Procedures to limit physical access to Pll and the facility or facilities in which 
they are housed while ensuring that properly authorized access is allowed, including physical 
barriers that require electronic control validation (e.g., card access systems) or validation by human 
security personnel. 

e. Physical Identification Validation: Access is physically safeguarded to prevent tampering and theft, 
including procedures to address control and validation of a person's access to facilities based on 
his or her need for access to the Pll. 

f. Operational Environment: Procedures that specify the proper functions to be performed, the 
manner in which they are to be performed, and the physical attributes of the surroundings of 
facilities where Pll is stored. 

g. Media Movement: Procedures that govern the receipt and removal of hardware and electronic 
media that contain Pll into and out of a facility. 

h. Encryption and Final Disposition of Information: Procedures addressing encryption of all data at 
rest and in transit and the final disposition of Pll. Procedures must include processes for the 
continued encryption of Pll through the time when its secure deletion/destruction has been 
requested by the Department and the Pll is deleted/destroyed. 

3. Technical Safeguards 

a. Data Transmissions: Technical safeguards, including encryption, to ensure Pll transmitted over an 
electronic communications network is not accessed by unauthorized persons or groups. 

b. Data Integrity: Procedures that protect Pll maintained by the Department or the Contractor from 
improper alteration or destruction. These procedures will include mechanisms to authenticate 
records and corroborate that they have not been altered or destroyed in an unauthorized manner. 
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c. Logging off Inactive Users: Inactive electronic sessions shall be designed to terminate 
automatically after a specified period of time. 

B. Security Controls Implementation 

The Department and the Contractor will develop procedures addressing the acquisition and operation of 
technology, the specific assignment of duties and responsibilities to managers and staff, the deployment of 
risk-appropriate controls, and the need for management and staff to understand their responsibilities and 
have the knowledge, skills and motivation necessary to fulfill their duties. 

C. Security Monitoring 

In combination with periodic security risk assessments, the Department and the Contractor will use a variety 
of approaches and technologies to make sure that risks and incidents are appropriately detected, assessed 
and mitigated on an ongoing basis. The Department and the Contractor will also assess on an ongoing 
basis whether controls are effective and performing as intended, including intrusion monitoring and data loss 
prevention. 

D. Security Process Improvement 

Based on the Department's security risk assessments and ongoing security monitoring, the Department and 
the Contractor will gather and analyze information regarding new threats and vulnerabilities, actual data 
attacks on the Department and the Contractor, and new opportunities for managing security risks and 
incidents. The Department and the Contractor will use this information to update and improve their risk 
assessment strategy and control processes. 

E. Breach Remediation 

The Department and the Contractor strive to keep Pll maintained by the Department and the Contractor 
secure and to use reasonable administrative, technical, and physical safeguards to do so. The Department 
and the Contractor will maintain and update incident response plans that establish procedures to follow in 
case a breach occurs. The Department and the Contractor will also identify individuals within their 
respective organizations responsible for implementing incident response plans if a breach should occur. 

If the Department or the Contractor determine that a breach has occurred, when there is a reasonable risk 
of identity theft or other harm, or where otherwise required by law, the Department or the Contractor, as 
applicable, will notify affected parties as promptly as possible, and will cooperate as needed to enable 
compliance with all breach of confidentiality laws. 

The Department and Contractor employee are required to report as promptly as possible to the Department 
leadership (or his or her designee) and persons responsible for managing their respective organization's 
incident response plan any incident or threatened incident involving unauthorized access to or acquisition of 
Pll of which they become aware. Such incidents include any breach or hacking of the Department's 
electronic data system or any loss or theft of data, other electronic storage, or paper. As used herein, 
"Electronic Data System" means all information processing and communications hardware and software 
employed in the Department's or the the Contractor's business, whether owned by the Department or the 
Contractor, or operated by its employees or agents in performing work for the Department or the Contractor. 

F. Personnel Security Policy Overview 

The Department and the Contractor shall mitigate the risks posed by users of the Department- or the 
Contractor-maintained Pll by: 

1. Performing appropriate background checks and screening of new Department and Contractor 
personnel, in particular those who will have access to the Department-maintained Pll; 

2. Obtaining agreements from the Department and the Contractor internal users covering confidentiality, 
nondisclosure and authorized use of Pll, including Pll contained in or accessed through the 
Department; and 
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3. Providing training to support awareness and policy compliance for new hires and annually for all 
Department and Contractor personnel. 

V. ENFORCEMENT 

The Department and the Contractor will consistently enforce this Policy with appropriate discipline for its own 
employees. The Department and the Contractor, as applicable, will determine whether violations of this Policy 
have occurred and, if so, will determine the disciplinary measures to be taken against any director, officer, 
employee, agent or representative who violates this P olicy. 

The disciplinary measures may include counseling, oral or written reprimands, warnings, probation or 
suspension without pay, demotions, reductions in salary, or termination of service or employment, as well as 
criminal referral to law enforcement, if appropriate. 

Persons subject to disciplinary measures may include, in addition to the violator, others involved in the 
wrongdoing such as (a) persons who fail to use reasonable care to detect a violation, (b) persons who withhold 
material information regarding a violation, and (c) supervisors who approve or condone the violations or attempt 
to retaliate against employees or agents or representatives of the Department or the Contractor for reporting in 
good faith violations or violators. 

The Department also may take appropriate actions authorized under contract or by law regarding the 
Contractor's failure to comply with the terms of this Policy. It is noted that if the U.S. Department of Education 
finds that the Contractor has violated FERPA requirements related to disclosure, the Contractor may be 
debarred from access to Pll from the Department for at leastfive (5) years. 
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APPENDIXJ 

REQUIREMENTS FOR SCORING PERFORMANCE ITEMS 

This appendix supplements the ITN description of the procedures required for scoring constructed response items on 
Florida Standards Assessments in English Language Arts/Literacy (ELA/L) and Mathematics. The ITN requires that 
student responses for constructed response items ELA/L tests in grades 3 and 10 and Algebra 1 be scored by at 
least two readers (100% second scored) using imaging technology or other technological applications that are 
equally responsive to the test administration and reporting schedule. As a quality control measure, 20% of all other 
constructed response items for ELA/L and Mathematics tests will require at least two readers' scores. The 
Department desires to implement processes that are reliable and valid as well as efficient in terms of time and 
expenditures. Therefore, respondents should, at a minimum, follow the processes described in this appendix. 
Enhancements to these processes are acceptable when approved by the Department. For example, respondents 
may propose an alternate method or formula for second reads to ensure quality control of scoring constructed 
response items. 

J.l Overview of Handscoring of English Language Arts/Literacy and Mathematics Constructed 
Response Items 

Respondents should submit scoring rubrics to be used in conjunction with other training materials for 
scoring constructed-response items for all grades and assessments. Field testing of items occurs in the 
spring of each year with items embedded within the operational tests. Near the end of operational scoring, 
contractor staff will begin selecting responses from the field test sample that represent a full range of 
student work. These responses will be presented to educator committees in mid-spring at field test 
rangefinding meetings. From these meetings, complete training and qualifying sets are created to train 
scorers who will score a sufficient number of responses to yield reliable statistical data for each constructed- 
response item. Every effort must be made to expedite scoring of the field test responses in order to have 
the item statistics available to construct and produce the following year's operational tests. Based on the 
number of constructed response items field tested each spring, the Department will consider an additional 
fall field test rangefinder meeting for the purpose of preparing training materials for constructed response 
items that were not scored in the summer. The statistics for these items will be banked for possible use on 
future operational tests. After all rangefinder meetings are completed, a final scoring material review 
meeting between the Department and contractor staff will be conducted to finalize training, qualifying, and 
initial validity materials for scorer training. 

Depending on the measurement model for the ELA/L assessment, the Department will consider a stand- 
alone field test for ELA/L extended-response tasks. Respondents who propose a stand-alone field test 
should prepare field test sampling specifications detailing the sampling methodology. The Department 
prefers that such a field test, if proposed, take place in the December time frame of each year of 
administration. 

J .2 Conduct Field-Test and Operational Rangefinding Meetings for ELA/L and Mathematics 

Prior to the scoring of constructed-response items field tested during the spring administrations, the 
contractor will be responsible for organizing and implementing rangefinding meetings to select training 
papers for the tasks. The meetings take place in Tallahassee and will involve Florida educators, 
Department staff, and the contractor's scoring directors and assistant directors over a period of up to three 
weeks each spring and each fall for field-test rangefinder meetings and each fall for operational rangefinder 
meetings. For all educator meetings, the Department and contractor staff will hold concurrent meetings for 
subjects and grades whenever possible. 

The contractor's subject area lead scoring directors, subject/grade scoring directors, and assistant scoring 
directors will participate in meetings with up to 10 Florida educators and a Department staff member who 
will serve as chair. 
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The contractor's staff will maintain a record of all scores and will keep notes during the rangefinder selection 
process to document the rationale for assigning scores. Following the meeting, the contractor's staff will 
meet with Department staff to finalize the student responses to be used as rangefinders and to identify 
others that may be used as training, qualifying, and validity responses. The contractor's staff is also 
responsible for providing written annotations in a format approved by the Department for the training 
materials. Requirements for selecting validity sets are identified in the following section. Contractor staff is 
also responsible for developing as part of the scoring guides any additional scoring notes or criteria required 
to conduct accurate scoring of the constructed-response tasks. The Department will approve all selections, 
annotations, or other materials to be used in training. See Section 7.5.4.5 for associated tasks. 

J .3 Conduct Scoring Material Review Meetings 

Meetings to review proposed scoring materials for operational tests will be conducted in Tallahassee at least 
three months prior to the beginning of handscoring training for the spring test administrations. The 
contractor's subject area lead scoring director and the subject/grade scoring directors will participate in 
meetings with Department staff to review scoring criteria and rangefinder papers that were used to score the 
tasks during field test scoring, or during the previous operational use. 

The purpose of each review meeting is to review the criteria, supplement the initial set of rangefinder papers 
with additional papers, if necessary, and to ensure that the contractor's scoring directors and Department 
staff share the same, detailed understanding of the scoring criteria for the operational constructed-response 
tasks. The scoring standards established by the initial field-test rangefinder selection will be maintained 
during the subsequent review. See Section 7.5.4.5 for associated tasks. 

J .4 Produce Scorer Training Materials 

Department staff will work closely with the contractor to prepare scoring materials. Frequent communication 
between Department and contractor staff can be expected following rangefinder selection and scoring 
material review meetings to initiate and complete the development of scoring materials. All scoring material 
will be submitted to the Department for review and approval according to an agreed-upon schedule. 
Scoring materials must be approved at least four weeks prior to the beginning of training and scoring. The 
contractor will be responsible for developing a detailed schedule to be included in the handscoring 
specifications identifying steps and schedules in the development of scoring materials. The contractor is 
responsible for producing the following scoring materials for each operational and field-test constructed- 
response task. 

• Scoring guides 

• Training sets 

• Qualifying sets 

• Validity sets 

• Daily calibration sets 

• Targeted calibration 

The contractor will develop a system for cataloging and storing all scoring materials developed during the 
course of the project. 

Scoring Guides - Scoring guides will be produced for each reader to use during training and scoring. The 
scoring guides will contain the rubrics, scoring criteria, constructed-response tasks, passages (if applicable), 
glossary of key terms, and other scoring guidelines designated by the Department and the contractor. The 
contractor will be responsible for writing annotations for the scoring guides, and the Department will give 
final approval to all annotations. 

Training Sets - Contractor must develop a sufficient number of training (practice) sets for ELA/L and 
Mathematics constructed-response items for scorer training to ensure valid and reliable scores for the 
duration of the project. The Department will work with contractor staff to determine the number and types of 



Florida Standards Assessments, ITN 2014-47 



138 



training sets to be used. The contractor will be responsible for writing the annotations for the training papers 
for all content areas, and the Department will give final approval to the annotations. The respondent should 
propose the process for administering the training sets. 

Validity Papers - Initial validity responses are selected from student responses to field-test administrations 
and do not appear in any other scoring materials. These responses allow the Department and contractor to 
monitor scoring accuracy and to maintain a consistent focus on the established rubrics and scoring 
guidelines. A sufficient number of initial validity responses must be approved in time to be loaded into the 
electronic scoring system before the scoring session begins. Validity responses are administered to readers 
online as part of the electronic scoring system and must be indistinguishable from live responses. The 
validity insertion rate will be agreed upon during the negotiation phase; however, for cost purposes, 
respondents should assume a validity insertion rate of 1 in 10 for extended ELA/L items and 1 in 25 for all 
other constructed response items. Once handscoring begins, scoring directors must refresh the validity pool 
during live scoring by escalating proposed validity responses to the Department for approval. All proposed 
validity papers for constructed-response items must have two agreement scores assigned. These scores 
may be generated by two scorers or by one scorer and a back-reading score. The frequency distribution of 
approved validity papers in the pool throughout the project should mirror the operational frequency 
distribution to the greatest extent possible. Responses that are approved by the Department will be moved 
into the electronic scoring system for random distribution to scorers. To avoid validity responses being 
repeatedly seen by the same scorers, adding and deleting validity papers within the electronic system is a 
continuous process throughout the project. 

The contractor will describe the estimated number of readers to be employed and plans for the 
administration of validity responses. 

Daily Calibration Sets - In addition to review of selected papers from the Scoring G uide at the start of each 
shift, daily calibration sets will be sent via the electronic scoring system at least once a day - usually after 
lunch or break. These 2-3 paper sets will be proposed by scoring directors and approved by the 
Department. The purpose of daily calibration is to highlight scoring issues or remind scorers of what to look 
for at a certain score point. Following the scoring of the calibration sets, the scoring directors will conduct a 
short discussion to further explain the scoring standards therein. 

Targeted Calibration Sets - Targeted calibration sets consist of 2-3 responses that target an individual's 
scoring trend and may be used as individual training/remediation for scorers whose quality metrics are 
below the standard for any reason. 

At the completion of scoring, the contractor will securely ship to the Department an electronic file of all 
scoring materials prepared for and utilized during scoring. 

J .5 Scoring Directors for Handscoring 

The contractor will assign its most qualified scoring staff to be scoring directors for Florida's Standards 
Assessments. All scoring directors must have worked in scoring director roles on a regular, continuing 
basis. The project and site scoring directors must be highly qualified to serve in these positions. They must 
have an appropriate educational background and extensive experience in directing state-level performance 
scoring projects as members of the contractor's regular scoring staff. The contractor will submit resumes of 
all handscoring staff to the Department for approval before assigning them to the project. 

The contractor will appoint a full-time project director to serve as overall director for Florida Standards 
Assessment handscoring activities. The project director must be available on a daily basis to discuss issues 
with the site scoring directors and the Department either in person, by e- mail, by phone or fax throughout 
the training and scoring sessions. 

The contractor must provide at least one full-time lead scoring director for each content area with expertise 
in their respective subject. This contractor staff will be available as liaison for Department content specialists 
for all handscoring activities. 
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The contractor will provide a site scoring director at each site who is dedicated solely to scoring of Florida's 
tests during the scoring period. Site scoring directors will be on site throughout the training and scoring 
sessions and will personally assist the scoring directors during the training of team leaders and readers and 
throughout the scoring sessions. 

The project director must be a senior scoring manager for the contractor. Each site scoring director must 
have served as a site scoring director for at least two projects in the past three years involving projects 
similar in size and scope to Florida's. 

The scoring directors who will direct the scoring for each subject/grade combination must be highly qualified 
to serve in this position. They must have an appropriate educational background and must have had 
experience serving as scoring directors at this level for previous scoring projects conducted by the 
contractor. The scoring directors must participate in the field test and operational rangefinder and scoring 
material review meetings. At least two scoring directors will be required for each grade and subject in order 
to be available in the event of the scheduling of two shifts or two scoring rooms. For the ELA/L extended 
constructed response items, it may be necessary to assign three scoring directors for this project. 

At least one assistant scoring director will be appointed to assist each scoring director in all aspects of 
directing scoring. The assistant scoring directors must also have an appropriate educational background 
and must have had experience serving as scoring directors or extensive experience serving as team leaders 
for previous scoring projects conducted by the respondent. The assistant scoring directors must also 
participate in all rangefinder meetings. 

Lead scoring directors, scoring directors, and assistant scoring directors must have their resumes approved 
by the Department prior to assignment to the project. 

As stated above, Florida's handscoring involves selection of rangefinders for field-test tasks at the same 
time that responses to the operational test are being scored. The scoring director and the assistant scoring 
director for each subject/grade will work together as a team to ensure that field test rangefinder selection 
and ongoing direction of operational scoring are both conducted at the highest levels of quality. It may be 
required that contractor staff assigned to the project be present at field test rangefinding meeting while other 
contractor staff remains at the scoring site to oversee completion of operational scoring. 

For each assessment, the project and site scoring directors will ensure that all additional training required 
for scoring is conducted prior to team leader training. Respondents should describe how they intend for this 
training to occur. 

J .6 Team Scoring Leaders 

The contractor will hire one team leader for every 10 to 12 readers. The team leaders may be especially 
recruited by the contractor or identified through the reader recruitment process. Team leaders must go 
through the same screening process as readers. The Department requires that team leaders have previous 
experience as readers and as team leaders if at all possible. At a minimum, team leaders must be 
experienced readers. 

Extensive training sessions approximately four days in length will be conducted for team leaders before 
readers are trained. The scoring directors and assistant scoring directors for each set of constructed- 
response tasks will conduct the training. The team leader training sessions will include the successful 
completion of a minimum of three training sets and two qualifying rounds. For ELA/L extended constructed- 
response writing tasks, Florida currently requires five training sets and three qualifying rounds. 

The initial training will include the training and qualifying procedures that will be used for the readers. In 
addition, there will be training on managing interaction with scorers, interpreting scoring reports, completing 
forms, and following all procedures required by the contractor and the Department. 
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J .7 Recruit and Hire Readers 

Respondents should include an analysis of the number of people that must be recruited, hired, and 
subsequently qualified as readers to complete the scoring within the time required to return reports to 
districts by the dates designated in the project schedule. The analysis must include the number and percent 
of people who will be expected to qualify, expected reader-attrition rate during each scoring session, and 
production required by reader and, if necessary, by shift at each site to complete the work. The respondent 
should describe the number of team leaders needed for each subject/grade. This detailed analysis should 
be completed in the reply for the Florida Standards Assessments in English Language Arts/Literacy and 
Mathematics. Respondents should also indicate expected changes in reader numbers across successive 
administrations. 

All readers are required to have earned a bachelor's degree in mathematics or the area of English 
Language Arts, or some related field. Readers of mathematics constructed-response tasks must have 
either been certified to teach the level of mathematics being assessed, have a bachelor's degree in 
mathematics education or mathematics, or must have completed a degree program with a sufficiently strong 
emphasis in mathematics to have the mathematics knowledge needed to effectively score responses to 
mathematics items. 

Potential readers in Florida may include retired Florida teachers or substitute teachers in the state of Florida 
provided that they will not teach in Florida's public schools (or work as a substitute teacher therein) for the 
remainder of the current school year. The Respondent is responsible for conveying this information and 
obtaining signed verification that potential readers agree to adhere to this stipulation. 

The contractor must work with the Department to implement a screening process for hiring readers to satisfy 
the Department's requirements. Potential readers must document that they meet the minimal education 
requirements during the screening process. Respondents should describe the screening process to select 
scorers. 

Potential readers with related degrees are those with degrees that involve course work and experience 
related to the content areas being scored. Potential candidates for training and qualifying to score 
mathematics shall have degrees in fields such as engineering, computer science, or liberal arts with an 
emphasis in mathematics. Likewise, for scoring English Language Arts/Literacy, candidates shall have 
degrees in fields such as English, English education, elementary education, reading education, history, 
psychology, or journalism. Potential readers will provide information about degree, certification, and course 
work taken. Minimum college coursework that represents a sufficiently strong emphasis in mathematics 
needed to score in this area is college-level credit in at least three courses to score at the high school level, 
two courses for the middle-school level, and one course to score at the elementary level. The contractor will 
provide for an independent, third-party verification that the potential reader has earned the identified degree. 
It has been the Department's experience that obtaining third-party verifications is more expensive and time- 
consuming for potential readers holding degrees conferred by foreign universities and degrees conferred by 
U.S. colleges and universities fifteen or more years ago. The contractor must take this into consideration 
when hiring potential readers. Potential readers whose degrees cannot be verified by a third party before 
the start of live scoring must be dismissed. No scores must be entered or maintained in the scoring system 
from readers with unverified qualifications. 

Potential readers also include those with a bachelor's degree or higher who have successfully completed a 
scoring project in the content area and at the elementary, middle, or high-school level being scored, 
qualifying and performing at levels required by Florida's program. Potential readers will ultimately 
demonstrate their qualification to score through successful completion of training and meeting the qualifying 
standard on scoring qualification rounds, as indicated in the Handscoring Specifications document. 

The contractor will document qualifications of readers assigned to be trained in an electronic file (the 
"Qualification File") specifying degrees earned; relevant teaching, educational, or work experience; and 
previous scoring experience relevant to scoring Florida's assessments. The Qualification Files will not 
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include personally identifiable information such as name, address, and telephone number. Documentation 
will be provided to Department staff, as needed, as it is accumulated throughout the hiring and training 
period. After scoring has begun, the contractor will only maintain Qualification Files for readers who 
successfully qualify and score student responses to Florida's assessments. 

The Department will monitor adherence to the screening process, review reader files as needed, approve 
acceptable related degrees, and provide general guidance on the screening process. Typically, each 
scoring site will be monitored by at least one representative from the Department for two to three weeks 
beginning with training. Prior to and during this on-site monitoring, the Department will provide oversight 
and monitoring of both potential Team Leader and reader qualifications. This monitoring will include a 
review for acceptability of potential readers for assignment to the handscoring program, 

In the month prior to handscoring training, weekly conference calls between the contractor and the 
Department will provide discussions surrounding site readiness, hiring/staffing, system readiness, and other 
pertinent topics related to handscoring. After recruitment of potential readers is complete, the Department 
staff will review a sample of potential readers' qualifications. Staff review of qualifications will be conducted 
utilizing a random sample of the Qualification Files. Staff will select additional potential scorer IDs for review 
as deemed necessary, e.g., for candidates who are having little success in scoring accurately during 
training. All qualifications for team leaders and a minimum of one in five candidates' documentation will be 
reviewed (e.g., for an initial hiring of 150 candidates for training, approximately 30 sets of documentation will 
be requested). Once scoring training begins, daily conference calls between the contractor and the 
Department will be conducted for project management purposes. 

As part of the screening process, all potential readers must sign a statement indicating that they understand 
the following conditions: 

• Applicants who do not successfully complete the training and qualifying requirements, they will not be 
hired as readers. 

• Hired readers may be dismissed if, after being trained to score, their scoring performance does not 
meet the requirements of the Department or the respondent acting on behalf of the Department. 

• Readers who are absent from scoring for two or more consecutive scoring days must repeat the 
training and qualifying procedures described above. 

• Applicants must agree that they will not teach or work as a substitute teacher in Florida's public 
schools for the remainder of the current school year. 

Applicants hired for training must also sign an agreement with the Department that they will maintain the 
security of Florida's test materials in addition to security agreements required by the contractor. 

J .8 Training and Qualifying of Readers 

The contractor will conduct separate training sessions for each constructed-response task or task set. 
Training is provided separately for each grade level. The scoring director will conduct training with the 
assistance of team leaders, under the direction of site scoring directors. The purpose of the training is to 
ensure that each person who scores Florida's student responses has met the Department's standards for 
scoring. The training process is essential for ensuring that scores assigned to performance task responses 
provide valid and reliable information. During training, the trainees will be provided with a scoring guide, 
highlighters and pens or pencils to make notations on training materials. At the conclusion of training, 
qualified readers will be taught how to use the alert system to identify students whose responses indicate 
the need for intervention. The contractor is responsible for developing training procedures in consultation 
with the Department. The Department will have final approval on all training techniques. 

To qualify as a reader, candidates must meet the established standards for the qualification sets approved 
by the Department. 

Assignment of scorers to items will be done in consultation with Department staff. 
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J .9 Handscoring Reports 

The contractor will collect reader performance statistics and provide electronic access to daily and 
cumulative reader handscoring reports to team leaders and above, including Department staff. The reports 
must include both real-time and partial- to full-day reports. Real-time reports must include reader inter-rater 
reliability and validity reports. The same reports are required for both operational and field-test scoring. The 
Department will determine, in consultation with the contractor, which of these reports will be available to the 
Department in real-time. The reports listed below are described generically. The contractor and the 
Department will work together initially to determine the format of this information that should be used for 
reporting to the Department. An annual review of the format and function of the reports will be conducted 
and some modifications may be required prior to the next test administration requiring handscoring based 
upon that review. The Department will have final approval on the design and user access level of each 
report. 

At a minimum, reports containing the following information will be produced and will include both numbers 
and percentages, as appropriate. 

1. Cumulative Training Report by Reader. This report identifies, by performance task or performance 
task sets, by reader, by team, and by site (subject/grade combination), performance on training sets. 

2. Cumulative Training Report by Document. This report identifies and summarizes, by performance task 
or performance task set, by team, and by site, performance on documents. 

3. Cumulative Qualifying Report by Reader. This report identifies, by performance task or performance 
task set, by reader, by team, and by site, performance on qualifying sets. 

4. Cumulative Qualifying Report by Document. This report summarizes, by performance task or 
performance task set, by team, and by site, performance on documents. 

5. Cumulative Reader Performance Report. This report includes the reader's scores on each training set, 
qualifying set, and validity set; daily inter-rater reliability statistics; and daily score point distribution. 
This report will be updated daily. The report will be accessed on request of the scoring director or the 
team leader. 

6. Daily and Cumulative Score Point Distribution. This report summarizes, by performance task or 
performance task set, by team, by site, and by rubric (0-2, 0-4, and 0-6), daily and cumulative 
distributions of score-point assignments. 

7. Daily and Cumulative Validity. This report summarizes, by performance task or performance task set, 
by team, and by site, daily and cumulative performance on validity sets. 

8. Daily and Cumulative Inter-rater Reliability. This report summarizes, by performance task or 
performance task set, by site, by team, and by rubric, daily and cumulative levels of inter-rater 
agreement statistics. 

9. Score Point Distribution for Standard Curriculum Students. This report summarizes, by team and by 
site, the cumulative distribution and mean of score-point assignments for standard curriculum students, 
for writing performance tasks only. 

10. Daily and Cumulative Score Point Distribution and Inter-rater Reliability by Reader. This report 
identifies and summarizes, by performance task or performance task set, by site, by team, by rubric, by 
reader, daily and cumulative score-point assignment distributions and levels of inter- rater agreement 
statistics (including perfect, adjacent, and disparate agreement; high/low scoring summaries; and 
required resolutions). 

11. Daily and Cumulative Validity by Reader. This report summarizes, by performance task or 
performance task set, by site, by team, by rubric, by reader, statistics indicating daily and cumulative 
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levels of agreement with validity set scores (including perfect, adjacent, and disparate agreement and 
high/low scoring summaries). 

12. Daily and Cumulative Validity by Validity Set. This report summarizes, by site, by team, by validity set, 
by reader, statistics indicating daily and cumulative levels of agreement with validity set scores 
(including a listing of validity set scores with readers assigned scores; perfect, adjacent, and disparate 
agreement; and high/low scoring summaries). 

13. Daily and Cumulative Read-Behind Reports. This report summarizes, for image scoring applications, 
by performance task or performance task set, by team, and by site, the result of online read-behinds by 
team leaders or scoring directors. 

14. Daily and Cumulative Timing Report. This report summarizes, by site, by team, by performance task 
or performance task set, the daily and cumulative number of readers engaged in scoring and statistics 
on actual time spent scoring performance tasks or performance task sets (including mean and range). 

15. Daily and Cumulative Completion Report. This report by site and by team, summarizes the daily and 
cumulative number of student responses read and the proportion of student responses remaining to be 
read. This report also compares on a daily and cumulative basis the amount of readings completed in 
comparison to projected completion targets. 

By the time of the first handscoring planning meeting with the Department, the contractor will provide an 
annotated list and samples of all reports, including reports used internally by the contractor to monitor the 
quality and pace of the scoring session. The Department may request that some internal reports be used 
for Florida and available to Department staff. 

At the completion of operational and field-test scoring, the contractor will provide the Department with final 
summaries of all cumulative handscoring reports. The summary reports are to be made available as 
electronic files and will be posted to a secure transfer file system maintained by the contractor and shared 
with the Department to access, manage, and transfer files as needed throughout the project. The contractor 
will produce a technical report that summarizes the score reports and provides details related to the 
reliability and validity of the field-test and operational handscoring. These materials will be reviewed at an 
annual handscoring debrief. 

J .10 Scoring Student Responses 

The rubrics used for all constructed-response tasks must be approved by the Department and will be used 
in conjunction with approved scoring guides and additional training materials throughout the handscoring of 
student responses. Constructed-response items for ELA/L tests in grades 3 and 10 and Algebra 1 will be 
scored by at least two readers (100% second scored) using imaging technology or other technological 
applications that are equally responsive to the test administration and reporting schedule. As a quality 
control measure, 20% of all other constructed response items for ELA/L and Mathematics tests will require 
at least two readers' scores. Responses receiving first and second reads of an individual student's response 
to an item will be randomly distributed across all readers designated to score that item. Assignment of 
responses across schools must be randomized to the extent that an individual reader scores responses 
from several schools within the same time period and does not score responses from the same school in 
succession. 

The Department will provide the contractor with a specific set of scoring rules for resolving disagreement 
between first and second readings with third and, infrequently, fourth readings and for calculating final 
scores. The following should be noted: 

• Third, independent readings are conducted by team leaders. 

• Complex resolutions and resolutions involving fourth readings and unscorable decisions are resolved 
by scoring directors. 

• All discrepancies for short-response tasks shall be resolved by a third reader 
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Attachment 1 
ITN 2014-47 Florida Standards Assessments 



RESPONDENT: 



We propose to provide the services being solicited for ITN 2014-47 Florida Standards Assessments. All work shall be performed in accordance with the specifications of this ITN, which has been reviewed and 
understood. The below prices are all inclusive. Payment for services will be made based on the deliverables and dates specified. There shall be no additional costs charged for work performed 
under this ITN, unless changes are made during the negotiation phase. 

Respondents may submit bids on any individual assessment components (A, B, C or D), any group of assessment components, or to provide all assessment components (A, B, C and D). The prices on this 
sheet will be used for evaluation purposes. All accommodations and deliverable costs should be included in each component price. In addition, Respondents should complete the pricing per deliverable in 
SECTION 7.8, for each assessment component, which will be used for negotiation purposes and any resulting contract. 





PRICING 


ASSESSMENT COMPONENTS 


YEAR 1 


YEAR 2 


YEAR 3 


TOTAL ORIGINAL 
CONTRACT TERM 
(YEARS 1-3) 


RENEWAL YEAR 
1 


RENEWAL YEAR 
2 


RENEWAL YEAR 
3 


GRAND TOTAL 
(ORIGINAL TERM +ALL 
RENEWALS)* 


A. Computer-based and paper-based assessments in English 
Language Arts/ Literacy in Grades 3-8 per schedule in 
Appendix A. 


















B. Computer-based and paper-based assessments in 
Mathematics in Grades 3-8 per schedule in Appendix A. 


















C. Computer-based and paper-based assessments in English 
Language Arts/Literacy in Grades 9-11 per schedule in 
Appendix A. 


















D. Computer-based and paper-based End-of-Course (EOC) 
assessments in Algebra 1, Geometry, and Algebra 2 per 
schedule in Appendix A. 


















E. All assessments (A-D) per schedule in AppendixA. 



















♦Evaluation points will be calculated on these costs. 



Vendor Name: 

Street Address: 

City, State, Zip Code: 

Telephone #: 

Fax: 

Authorized Agent (typed): 
Authorized Agent (Signature): 
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ATTACHMENT 2 
DRUG-FREE WORKPLACE 

(will be considered in case of identical tie replies) 

Preference shall be given to businesses with drug-free workplace programs. Whenever two or more bids which are 
equal with respect to price, quality, and service are received by the state or by any political subdivision for the 
procurement of commodities or contractual services, a bid received from a business that certifies that it has 
implemented a drug-free workplace program shall be given preference in the award process. Established 
procedures for processing tie bids will be followed if none of the tied vendors have a drug-free workplace program. In 
order to have a drug-free workplace program, a business shall: 

1) Publish a statement notifying employees that the unlawful manufacture, distribution, dispensing, possession, or 
use of a controlled substance is prohibited in the workplace and specifying the actions that will be taken against 
employees for violations of such prohibition. 

2) Inform employees about the dangers of drug abuse in the workplace, the business's policy of maintaining a drug- 
free workplace, any available drug counseling, rehabilitation, and employee assistance programs, and the 
penalties that may be imposed upon employees for drug abuse violations. 

3) G ive each employee engaged in providing the commodities or contractual services that are under bid a copy of 
the statement specified in subsection (1). 

4) In the statement specified in subsection (1), notify the employees that, as a condition of working on the 
commodities or contractual services that are under bid, the employee will abide by the terms of the statement 
and will notify the employer of any conviction of, or plea of guilty or nolo contendere to, any violation of Chapter 
893 or of any controlled substance law of the United States or any state, for a violation occurring in the 
workplace no later than five (5) days after such conviction. 

5) Impose a sanction on, or require the satisfactory participation in a drug abuse assistance or 
rehabilitation program if such is available in the employee's community, by any employee who is so 
convicted. 

6) Make a good faith effort to continue to maintain a drug-free workplace through implementation of this 
section. 

As the person authorized to sign the statement, I certify that this firm complies fully with the above 
requirements. 



Vendor's Signature 
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ATTACHMENT 3 



STATEMENT 



DISCLOSURE 



PARTNERSHIP OR INDIVIDUAL 

I hereby certify that I, if an individual, or each of us, if a partnership, doing business as of , 

(Name of Individual or Partnership) 

(am)(is) not now involved in nor have I ever engaged in any private business venture or enterprise, directly or indirectly, with 
the Commissioner of Education, the Deputy Commissioner of Education, any Associate Commissioner of Education, 
Division Director, or Bureau Chief within the Florida Department of Education. 

I further certify that neither I, nor any partner, if a partnership, nor anyone acting in my or our behalf has requested that any 
of the above designated persons or any other employee of the Department of Education exert any influence to secure the 
appointment of under this proposed agreement. 

(Name of Individual or Partnership) 

(1) 

Signature 



Signature 

(1) If partnership, each partner must sign and execute. 

Signature 



COMPANY OR CORPORATION 

I hereby certify that neither I nor any owner, officer, director, or shareholder of , a 

(Name of Corporation/Company) 

(1) corporation, licensed to do business in Florida, is presently involved in or has 

(Name of State of Inc.) 

been engaged in any private business venture or enterprise, directly, or indirectly, with the Commissioner of Education, the 
Deputy Commissioner of Education, any Associate Commissioner of Education, Division Director, or Bureau Chief within the 
Florida Department of Education. 

I further certify that neither I nor any owner, officer, director, or shareholder of this corporation or anyone acting on behalf of 
this corporation or any of its owners, officers, directors, or shareholders has requested that any of the above designated 
persons or any other employee of the Department of Education exert any influence to secure the appointment of 
under this proposed agreement. 

(Company /Corporation) 

(2) 

Signature 



Title 

(1) If company is not incorporated, insert "not incorporated" in this space. 

(2) If incorporated, this statement is to be executed by same person who will execute Contract, if awarded. 
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ATTACHMENT 4 
REFERENCES 



RESPONDENT NAME: 



PROVIDE THE FOLLOWING REFERENCE INFORMATION FOR A MINIMUM OF THREE (3) BUSINESSES WHERE 
SERVICES OF SIMILAR SIZE AND SCOPE HAVE BEEN COMPLETED. 



BUSINESS NAME: 




ADDRESS: 




CONTACT PFRSON' 

v^u 111 i nv^ i r LrvjuN, 




PHONE NUMBER' 




FAX NUMRFR' 

1 nA 111 U 1*1 U L l \ i 




F-MAII ADDRFSS' 

l_ I'lnlL nUUI\L J J i 




DATF AND DESCRIPTION 

Ur\ 1 L r\IHL/ L/ 1_ J [\ 1 r 1 IUI1I 

OF SERVICES: 




BUSINESS NAME: 




ADDRESS: 




CONTACT PERSON' 

v_. \J I \ I nL I r Li\Juiiii 




PHONE NUMBER' 

I MWIVL 1 V U 1*1 U l_ l\ ■ 




FAX NUMBER' 

1 nA 111 UI'lULIli 




E-MAIL ADDRESS' 




DATF AND DFSCRIPTION 

L/n 1 l_ rA 1 M L/ UL JLIMT 1 lull 

OF SERVICES: 




BUSINESS NAME: 




ADDRESS: 




CONTACT PERSON: 




PHONE NUMBER: 




FAX NUMBER: 




E-MAIL ADDRESS: 




DATE AND DESCRIPTION 
OF SERVICES: 
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ATTACHMENT 5 
MINORITY SUB CONTRACTORS UTILIZATION SUMMARY 

STATE OF FLORIDA DEPARTMENT OF EDUCATION 

The Department's Supplier Diversity initiative strives to ensure the promise of Florida's future is shared by all of its 
residents, regardless of race, ethnicity, disability, neighborhood or background. To that end, the Department is 
dedicated to support, track and increase its small, minority-, women-, and service-disabled veteran business 
enterprise spending with Prime Contractors and subcontractors. This form was developed to assist in these efforts. 

The Prime Contractor shall report all small, minority-, women-, and service-disabled veteran business enterprise 
Subcontractors, identifying the Name, Address, Type of Certification and Dollar Amount on the form below. The 
Prime Contractor shall submit this form with each invoice submitted for payment, whether or not funds have been 
spent with a small, minority-, women-, and service-disabled veteran business enterprise subcontractor for the period 
covered by the invoice. The Office of Supplier Diversity, Florida Department of Management Services will assist in 
furnishing names of qualified minorities. The Office of Supplier Diversity can be reached at (850) 487-0915; the 
Internet Web address is http://dms.myflorida.com/other programs/office of supplier diversity osd . 



PRIME CONTRACTOR: 
CONTRACT NO.: 



CONTRACT TITLE: 



MBE CONTRACTORS 
Full Name, Address, Telephone Number 


State 
Certified 


Non- 
Certified 


Non- 
profit 


Dollar 
Amount 





















































Total Amount $ 



Certified True and Correct by: Submit Report to: 

Mrs. J anice Brown 

Prime Contractor Bureau of Contracts, Grants & Procurement 

Management Services 

Title 325 West Gaines Street 

332 Turlington Building 

Date ~ ~ Tallahassee, Florida 32399-0400 



For additional information, you may call Mrs. Brown at (850) 245-0819, or e-mail janice.brown@ fldoe.org 
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ATTACHMENT 6 
VENDOR CERTIFICATION REGARDING 
SCRUTINIZED COMPANIES LISTS 



Respondent Vendor Name: 


Vendor FEIN: 




Vendor's Authorized Representative Name and Title: 




Address: 




City: State: 


Zip: 


Phone Number: 




Email Address: 







Section 287.135, Florida Statutes, prohibits agencies from contracting with companies, for goods or services over 
$1,000,000, that are on either the Scrutinized Companies with Activities in Sudan List or the Scrutinized Companies 
with Activities in the Iran Petroleum Energy Sector List. Both lists are created pursuant to section 215.473, Florida 
Statutes. 



As the person authorized to sign on behalf of Respondent, I hereby certify that the company identified above in the 
section entitled "Respondent Vendor Name" is not listed on eitherthe Scrutinized Companies with Activities in Sudan 
List or the Scrutinized Companies with Activities in the Iran Petroleum Energy Sector List. I understand that pursuant 
to section 287.135, Florida Statutes, the submission of a false certification may subject company to civil penalties, 
attorney's fees, and/or costs. 



Certified By: 



Who is authorized to sign on behalf of the above referenced company. 
Authorized Signature Print Name and Title: 
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ATTACHMENT 7 
CONTRACT STANDARD TERMS AND CONDITIONS 
STATE OF FLORIDA DEPARTMENT OF EDUCATION 

I. Pursuantto S. 287.058(1), Florida Statutes ("F.S."): 

A. Bills for fees or other compensation for services or expenses shall be submitted in detail sufficient for a proper preaudit and 
postaudit thereof. 

B. Travel expenses will be reimbursed only if expressly authorized by the terms of the Contract. Bills for any travel expenses 
shall be submitted in accordance with s. 112.061, F.S. 

C. The Department may unilaterally cancel this Contract if the Contractor refuses to allow access by members of the public to all 
documents, papers, letters and materials made or received in conjunction with the Contract that are subject to Chapter 119, 
F.S., and are not exempt from public inspection by s 119.071, F.S., or by other provisions of general or special law. 

D. The Deliverables specified in the Contract must be received and accepted in writing by the Department's Contract Manager 
before Contractor is entitled to payment. 

E. To complete this Contract, all services must be performed and/or goods received on or before the date(s) specified in the 
Contract. 

F. If th is Contract is expressly renewable, it may be renewed for a period that may not exceed three years or the term of the 
original contract, whichever is longer. The renewal price for the contracted service is set forth in the bid, proposal, reply. Cost 
for renewal shall not be changed. Renewals shall be contingent on satisfactory performance evaluations by the Department 
and subject to the availability of funds. Exceptional purchase contracts pursuantto s. 287.057(3)(a) and (c), F.S., may not be 
renewed. 

II. The Contractor shall prepare an invoice for the amount due and mail it to the Department of Education Comptroller after having 
delivered the products and services required under this Contract to the Contract Manager. The invoice shall set forth details 
sufficient for a proper pre-audit and post-audit including, where applicable, the products and services delivered and completion 
dates. Upon receipt of the invoice, the Department of Education Comptroller will request confirmation from the Contract Manager 
that the delivered products and services are satisfactory and payment is due. If for any reason they are not satisfactory, payment 
will be withheld until the unsatisfactory condition or conditions are corrected. Upon receipt of the Contract Manager's approval, the 
Department of Education Comptroller shall process each invoice in accordance with the provisions ofs. 215.422, F.S. 

A. Contractor agrees to submit invoice within thirty (30) days of the Departments acceptance of deliverables. It is understood 
that should Contractor fail to submit invoice within thirty (30) days following the Department's acceptance of the deliverables, 
the Department shall not be responsible for payment thereof under this contract or quantum meruit. 

III. Section 215.422, F.S., provides that agencies have five (5) working days to inspect and approve goods and services, unless bid 
specifications or the Contract specifies otherwise. With the exception of payments to health care providers for hospital, medical, or 
other health care services, if payment is not available within forty (40) days, measured from the latter of the date the invoice is 
received or the goods or services are received, inspected and approved, a separate interest penalty set by the Comptroller pursuant 
to s. 55.03, F.S., will be due and payable in addition to the invoice amount. To obtain the applicable interest rate, please contact the 
Departments Fiscal s. at 850/245-0401 or Purchasing Office at 850/245-0735. Payments to health care providers for hospitals, 
medical, or other health care services, shall be made not more than thirty-five (35) days from the date of eligibility for payment is 
determined, and the daily interest rate is .02740 percent. Invoices returned to a vendor due to preparation errors will result in a 
payment delay. Invoice payment requirements do not start until a properly completed invoice is provided to the agency. A Vendor 
Ombudsman, whose duties include acting as an advocate for vendors who may be experiencing problems in obtaining timely 
payment(s) from a State Agency, may be contacted at 866/352-3776 or by calling the Chief Financial Officer's Hotline, 800/342- 
2762. 

IV. As used in this Contract, the term "Deliverable" refers to tangible "commodities", as defined in s. 287.012(5), F.S., which the 
Contractor provides pursuant to the Contract and to reports or other tangible or documentary evidence which demonstrate that the 
Contractor has performed the services required by the Contract. The following provisions govern Deliverables, as applicable: 

A. Each Deliverable must be physically delivered to the Departments Contract Manager, or to a person designated by the 
Contact Manager. If delivery is made to a designee, the Contractor shall give written notice to the Contract Manager of the 
delivery. A Deliverable is not received until the Contract Manager has physical control of deliveries or has written notice that 
the designee has physical control. 

B. In each case in which the approval of a Deliverable is dependent upon tests being conducted by the Department or Contractor, 
independently or jointly, the Departments inspection and approval of the Deliverable shall not be subject to the five (5) day 
provision in s. 215.422, F.S., but shall be governed by the terms and conditions of the acceptance testing plan as stated in 
AttachmentA, until approved in accordance with the plan. 

C. In each case of a Deliverable of information technology, as defined at s. 287.012(14), F.S., unless specified otherwise in 
AttachmentA, the acceptance testing plan is deemed to include as a minimum the reliable performance of the information 
technology in accordance with its design specifications in: 

1. a test environment that simulates the production environment as much as is reasonably possible; and 

2. the production environment for which it is intended for a period of time sufficient for the information technology to have 
experienced the major foreseeable exigencies of the production functions. 

D. The Departments inspection, including testing when applicable, shall determine whether or not the Deliverables appear to be 
in compliance with the Contract. The Contractor shall be notified in writing of any apparent deficiency. The written notice shall 
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detail the specific action required by the Contractor to correct the deficiency. The Contractor shall timely correct such 
deficiency and resubmit the deliverable for acceptance. 

V. The Contractor represents and agrees that information submitted in support of its requests for payment is the basis of payment and 
is true and accurate to the best of knowledge of the responsible signatory. A violation of this provision shall subject the violator to 
the provisions of s. 68.082, F.S., pertaining to false claims against the State, and/or s. 837.06, F.S., pertaining to false official 
statements. 

VI. This paragraph applies if this Contract expires in a fiscal year subsequent to the fiscal year in which the Contract is entered. The 
State of Florida's fiscal year comprises J uly 1 through J une 30. The Department's and State of Florida's performance and obligation 
to pay under this Contract is contingent upon an annual appropriation by the Legislature. If the Legislature fails to make the 
necessary appropriation, the Department will determine if there are other unencumbered funds which are available and which can 
be lawfully expended to pay for the Departments obligations hereunder. If the Department determines that there are no such funds, 
the Department shall promptly notify the Contractor. The giving of notice shall be deemed to have cancelled this Contract by mutual 
consent, with the date of notice being the date of cancellation. 

VII. Notwithstanding anything to the contrary contained in a State Term Contract, Contractor warrants that all commodities, as defined in 
s. 287.012, F.S., shall meet the specifications of the Contract and shall be merchantable and fit for the particular purposes intended 
by the Contract. 

VIII. The Contractor further warrants that as to each Deliverable produced pursuant to this Contract, Contractor's production of the 
Deliverable, and the Departments use of the Deliverable, will not infringe on the copyrights of any third party. This provision applies 
to each work of authorship in which copyrights subsist pursuant to 17 U.S.C. Sections 102-105 and to each exclusive right 
established in 17 U.S.C. Section 106. In furtherance of this provision the Contractor additionally warrants that: 

A. As to each work of software or other Information technology", as defined in s. 287.012(15), F.S., in which copyrights subsist, 
the Contractor has acquired the rights by conveyance or license to any third party software or other information technology, 
which was used to produce the Deliverable; 

B. As to each image and sound recording incorporated into a Deliverable, the Contractor has acquired the necessary rights, 
releases, and waivers from the person whose image or sound is included, or from the holder of the copyrights subsisting in the 
literary, musical, dramatic, pantomime, choreographic, pictorial, graphic, sculptural, motion pictures, audiovisual work or sound 
recording from which the included image or sound recording was taken. 

IX. The Contractor further warrants that the Contractor shall not disclose to any third party, without the express, prior, written approval 
of the Department, any personally identifiable information about any student. This applies to information which came from any 
record or report of a Florida public education institution or from any education record which is subject to the Family Educational 
Rights and Privacy Act, 20 U.S.C. Section 1232g. The terms "record a report" and "student" shall have the meanings prescribed in 
s. 1002.22(2)(c) and (d), F.S. The term "educational record" shall have the meaning prescribed in 20 U.S.C. Section 1232g(a)(4). 

X. In the event that the Governor and Cabinet are required to impose a mandatory reserve on appropriations, the Department shall 
amend this Contract to place in reserve the amount determined by the Department of Education to be necessary because of the 
mandatory reserve. Such amendments may provide for adjustments in the Deliverable products and services as may be necessary. 

XI. Intellectual property is subject to following additional provisions: 

A. Anything by whatsoever designation it may be known, that is produced by, or developed in connection with, this Contract shall 
become the exclusive property of the of the State of Florida and may be copyrighted, patented, or otherwise restricted as 
provided by Florida or federal law. Neither the Contractor nor any individual employed under this Contract shall have any 
proprietary interest in the product. 

B. With respect to each Deliverable that constitutes a work of authorship within the subject matter and scope of U.S. Copyright 
Law, 17 U.S.C. Sections 102-105, such work shall be a "work for hire" as defined in 17 U.S.C. Section 101 and all copyrights 
subsisting in such work for hire shall be owned exclusively by the Department pursuant to s. 1006.39, F.S., on behalf the State 
of Florida. 

C. In the event it is determined as a matter of law that any such work is not a "work for hire", Contractor shall immediately assign 
to the Department all copyrights subsisting therein for the consideration set forth in the Contract and with no additional 
compensation. 

D. The foregoing shall not apply to any preexisting software, or other work of authorship used by Contractor, to create a 
Deliverable but which exists as a work independently of the Deliverable, unless the preexisting software or work was 
developed by Contractor pursuant to a previous Contract with the Department or a purchase by the Department under a State 
Term Contract. 

E. The Department shall have full and complete ownership of all software developed pursuant to the Contract including without 
limitation: 

1. The written source code; 

2. The source code files; 

3. The executable code; 

4. The executable code files; 

5. The data dictionary; 

6. The data flow diagram; 

7. The work flow diagram; 

8. The entity relationship diagram; and 
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9. All other documentation needed to enable the Department to support, recreate, revise, repair, or otherwise make use of 
the software. 

XII. The Department reserves the right, at its option, to issue a change order to delete work tasks reducing the total Contract amount by 
up to 10%. An addition of work tasks within the scope of the Contract, an increase in the total Contract amount, or a decrease of 
more than 10% of the total Contract amount, shall be implemented only by a Contract amendment signed by both the Department 
and the Contractor. 

XIII. Pursuant to s. 216.347, F.S., no funds awarded under this Contract may be used for the purpose of lobbying the Legislature, the 
judicial branch, or a State agency. 

XIV. The Contractor shall grant access to all records pertaining to the Contract to the Department's Inspector General, General Counsel 
and other agency representatives, the State Auditor General, the Office of Program Policy and Government Accountability, and the 
Chief Financial Officer. 

XV. The Contractor agrees to permit onsite visits by designated Department employees or agents to conduct audits to ensure 
compliance with Section 20.055, Florida Statutes. These audits may require Department access to records and data, computers and 
communications devices, and other materials whether owned or operated by the Contractor. Access may include, but is not limited 
to, user level and/or system level access to any computing or communications device; access to information (electronic, hardcopy, 
etc) that may be produced, transmitted or stored on the Contractor's equipment or premises; access to work areas; and access to 
interactively monitor and log traffic on the Contractor's networks. 

XVI. The Contractor must carry general liability insurance, which shall include errors and omissions coverage. The amount of coverage 
shall be a minimum of $1,000,000 or the aggregate total of all contractual agreements between the Contractor and the agencies and 
political subdivisions ofthe State of Florida, whichever is greater. The Contractor shall add the Department as an additional insured 
on the general liability coverage. The insurance shall cover all of the Contractor's operations under this Contract and shall be 
effective throughout the Term of this Contract, as well as any renewals or extensions thereto. It is not the intent of this Contract to 
limit the types of insurance otherwise required by this Contract or that the Contractor may desire to obtain or be required to obtain 
by law. The Contractor must submit a Certificate of Insurance indicating coverage for general liability purposes and additional 
insured coverage, and shall maintain and pay for same throughout the Term of this Contract. A Certificate of Insurance indicating 
adequate coverage shall be submitted to the Department prior to the time the Contract is entered. Any and all insurance policies 
shall be through insurers qualified to do business in Florida. 

XVII. The Contractor agrees to provide the Department upon execution of this Contract with a performance bond or other security 
deposited with the Department in the total amount of the Contract or another amount if specified in the procurement specifications or 
AttachmentA, guaranteeing thatthe Contractor will perform all work according to this Contract, within the time and price specified in 
the Contract. A performance bond shall be issued from a surety company, qualified to do business in Florida. 

XVIII. The Contractor may not assign or subcontract all or any portion of this Contract without the advance written consent of the 
Department. 

XIX. In all cases in which the Contractor, with the advance written consent of the Department, assigns or subcontracts, all or any portion 
ofthe Contract: 

A. The Contractor shall monitor the subcontractor or assignee and establish controls to avoid or mitigate risks identified by the 
Department or the Contractor; and 

B. The Contractor shall allow the Department to monitor subcontractor or assignee activity and compliance, and the Contractor 
shall require the subcontractor or assignee to promptly submit to the Department, at the Department's request, complete and 
accurate documentation pertaining to the subcontract or the Contract. 

XX. The Contractor shall coordinate with and assist the Departments Contract Manager in the performance of the latter's 
responsibilities, which include without limitation: 

A. Monitoring the activities ofthe Contractor; 

B. Receiving and reviewing the reports of the Contractor to determine whether the objectives of the Contract are being 
accomplished; 

C. Receiving and reviewing the invoices for payment of funds to assure thatthe requirements ofthe Contract have been met and 
that payment is appropriate; 

D. Evaluating the process used by the Contractor to monitor the activities of any subcontractor or assignee; and 

E. Accessing, directly, the subcontractors and assignees, as the Contract Manager deems necessary. 

XXI. This Contract may not be modified unless in writing signed by the Department and the Contractor. 

XXII. The Department and the Contractor waive application of the principle of contract construction that ambiguities are to be construed 
against a contract's drafter, and agree that this Contract is their joint product. 

XXIII. The Department and the Contractor acknowledge that they have had their respective attorneys review and approve this Contractor 
that they have had the opportunity to do so. 

XXIV. This Contract shall be governed by the laws of the State of Florida, and venue for purposes of any action brought to enforce or 
construe the Contract shall lie in Leon County, Florida. 

XXV. Failure of the Department to declare any default immediately upon the occurrence or knowledge thereof, or delay in taking any 
action in connection therewith, does not waive such default. The Department shall have the right to declare any such default at any 
time and take such action as might be lawful or authorized under the Contract, at law, or in equity. No Department waiver of any 
term, provision, condition or covenant of the Contract shall be deemed to imply or constitute a further Department waiver of any 
other term, provision, condition or covenant of the Contract, and no payment by the Department shall be deemed a waiver of any 
default under the Contract. 
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XXVI. Time is of the essence with regard to each and every obligation of the Contractor contained in the Contract. Each such obligation is 
deemed material, and a breach of any such obligation (including a breach resulting from the untimely performance thereof) shall 
constitute a material breach. 

XXVII. The Contractor shall indemnify and hold harmless the Department, its attorneys, agents and employees, from and against any and 
all third party claims, suits, debts, damages, and causes of action, whatsoever, whether arising in law or in equity, arising out of or 
relating to Contractor performance or failure to perform under this Contract. The indemnification shall include reasonable attorney 
fees and costs incurred by the Department, its attorneys, agents and employees, in the defense of any such claim, suits or causes 
of action, as aforesaid. 

XXVIII. This Contract may be cancelled by written agreement of the Department and the Contractor specifically referencing this Contract. 
Such agreement shall specify the remaining measures necessary to be taken by each party. 

XXIX. The Department reserves the right to cancel this contract without cause by giving the Contractor thirty (30) days written notice. 

XXX. Should Contractor fail to perform to Contract terms and conditions, Contractor shall be notified in writing, stating the nature of the 
failure to perform and providing a time certain (which shall be not less than ten (10) days following receipt of such notice) for 
correcting the failure. Such failure to perform shall otherwise be dealt within accordance with Rule 60A-1.006, F.A.C. 

XXXI. A person or affiliate who has been placed on the convicted vendor list following a conviction for a public entity crime may not submit 
a bid on a contract to provide any goods or services to a public entity, may not submit a bid on a contract with a public entity for the 
construction or repair of a public building or public work, may not submit bids on leases of real property to a public entity, may not be 
awarded or perform work as a contractor, supplier, subcontractor, or consultant under a contract with any public entity, and may not 
transact business with any public entity in excess of the threshold amount provided in s. 287.017, F.S., for CATEGORY TWO for a 
period of thirty-six (36) months from the date of being placed on the convicted vendor list. 

XXXII. The employment of unauthorized aliens by any contractor is considered a violation of Section 274A(e) of the Immigration and 
Nationality Act. If the contractor knowingly employs unauthorized aliens, such violation shall be cause for unilateral cancellation of 
the contract. In addition, pursuant to State of Florida Executive Order No. 11-116, Contractor shall utilize the U.S. Department of 
Homeland Security's E-Verify system to verify the employment of all new employees hired by the Contractor during the contract 
term. Also, Contractor shall include in related subcontracts a requirement that subcontractors performing work or providing services 
pursuant to the Contract utilize the E -Verify system to verify employment of all new employees hired by the subcontractor during the 
contract term. 

XXXIII. Certification Regarding Debarment, Suspension, Ineligibility and Voluntary Exclusion - LowerTier Covered Transactions 

A. The prospective lower tier participant certifies, by submission of this proposal, that neither it nor its principals is presently 
debarred, suspended, proposed for debarment, declared ineligible, or voluntarily excluded from participation in this transaction 
by any Federal department or agency. 

B. Where the prospective lower tier participant is unable to certify to any of the statements in this certification, such prospective 
participant shall attach an explanation to this proposal. The Department may cancel this contract if an attached explanation is 
not acceptable to the Department or the Federal government. 

XXXIV MyFloridaMarketPlace 

A. MyFloridaMarketplace Vendor Registration 

Each Vendor doing business with the State of Florida for the sale of commodities or contractual services as defined in section 
287.012, Florida Statutes, shall register in MyFloridaMarketPlace, in compliance with Rule 60A-1.030, Florida Administrative 
Code, unless exempt under Rule 60A-1.030(3) Florida Administrative Code. 

B. MyFloridaMarketplace Transaction Fee 

The State of Florida, through the Department of Management Services, has instituted MyFloridaMarketPlace, a statewide 
eProcurement system. Pursuant to section 287.057(23), Florida Statutes (2010), all payments for commodities and/or 
contractual services as defined in Section 287.012, Florida Statutes, shall be assessed a Transaction Fee of one percent 
(1.0%), which the Vendor shall pay to the State, unless exempt under Rule 60A-1.032, Florida Administrative Code. 
Notwithstanding the provisions of Rule 60A-1.030, et seq., the assessment of a transaction fee shall be contingent upon 
Federal approval of the transaction fee assessment program and continued payment of applicable federal matching funds. 

For payments within the State accounting system (FLAIR or its successor), the Transaction Fee shall, when possible, be 
automatically deducted from payments to the Vendor. If automatic deduction is not possible, the Vendor shall pay the 
Transaction Fee pursuant to Rule 60A-1. 031(2), Florida Administrative Code. By submission of these reports and 
corresponding payments, Vendor certifies their correctness. All such reports and payments shall be subject to audit by the 
State or its designee. 

The Vendor shall receive a credit for any Transaction Fee paid by the Vendor for the purchase of any item(s) if such item(s) 
are returned to the Vendor through no fault, act, or omission of the Vendor. Notwithstanding the foregoing, a Transaction Fee 
is non-refundable when an item is rejected or returned, or declined, due to the Vendor's failure to perform or comply with 
specifications or requirements of the agreement. 

Failure to comply with these requirements shall constitute grounds for declaring the Vendor in default and recovering 
reprocurement costs from the Vendor in addition to all outstanding fees. VENDORS DELINQUENT IN PAYING 
TRANSACTION FEES MAY BE EXCLUDED FROM CONDUCTING FUTURE BUSINESS WITH THE STATE. 
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